2024-09-21T23:01:27.961 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T23:01:27.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T23:01:28.059 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914511' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 1478 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 24/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVINPiRSwLqQnBj7mKrhyVes1+mXwAb/TjYPRWSp8QguwEf/1DEpP248nae+R374DfPV3QlMBtHi3xG6GU3fq0= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJ72l0rICXir5VJ/uMGGuBOISpYaveP3zwcYT1/tcD+yfFixB/bh5TsxJ/nk6r0dZaqSkwFOI1PY18VoXFZh7Y= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwXtcli5i3JH3EE0aBlbr+l1kgAJnFV+kbJ2ofzRgObTFnLEZXMrywjL5X79qd/vPuvoIHGKF6QYcu/Hafhi80= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T23:01:28.059 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-21T23:01:28.060 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-21T23:01:28.060 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T23:01:28.062 INFO:teuthology.task.internal:Checking packages... 2024-09-21T23:01:28.082 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-21T23:01:28.082 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T23:01:28.082 INFO:teuthology.packaging:ref: None 2024-09-21T23:01:28.082 INFO:teuthology.packaging:tag: None 2024-09-21T23:01:28.082 INFO:teuthology.packaging:branch: squid 2024-09-21T23:01:28.083 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:01:28.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-21T23:01:28.309 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:01:28.310 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T23:01:28.317 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T23:01:28.317 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T23:01:28.385 INFO:teuthology.task.internal:Saving configuration 2024-09-21T23:01:28.398 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T23:01:28.405 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T23:01:28.430 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 22:55:56.543010', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMz5cM9sDJdpmcL4Gk07/VnKMNC2dstCqLSmNaztTJug'} 2024-09-21T23:01:28.455 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 22:55:56.541635', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIMrbhGFb/2pdZlnefEtvhiAXZClldU8NHKPaikuyQJz'} 2024-09-21T23:01:28.474 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 22:55:56.544318', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBhsTXRvNYmfG6eRkx/eCuOQchSsiMfw0NfJItQdm2uJ'} 2024-09-21T23:01:28.474 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T23:01:28.480 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-21T23:01:28.480 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-21T23:01:28.480 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['client.0'] 2024-09-21T23:01:28.481 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T23:01:28.568 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa63dbaf880>, signals=[15]) 2024-09-21T23:01:28.568 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T23:01:28.574 INFO:teuthology.task.internal:Opening connections... 2024-09-21T23:01:28.574 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:01:28.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:01:28.656 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:01:28.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:01:28.734 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T23:01:28.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:01:28.812 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T23:01:28.819 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-09-21T23:01:28.824 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-09-21T23:01:28.825 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-09-21T23:01:28.871 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Ubuntu" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="22.04" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_CODENAME=jammy 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:ID=ubuntu 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE=debian 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T23:01:28.872 INFO:teuthology.orchestra.run.smithi053.stdout:UBUNTU_CODENAME=jammy 2024-09-21T23:01:28.873 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-09-21T23:01:28.894 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-09-21T23:01:28.899 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-09-21T23:01:28.900 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-09-21T23:01:28.944 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T23:01:28.944 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="22.04" 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=jammy 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T23:01:28.945 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=jammy 2024-09-21T23:01:28.945 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-09-21T23:01:28.969 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-09-21T23:01:28.975 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-09-21T23:01:28.975 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-09-21T23:01:29.020 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="22.04" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=jammy 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T23:01:29.021 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=jammy 2024-09-21T23:01:29.022 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-09-21T23:01:29.044 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T23:01:29.058 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T23:01:29.094 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T23:01:29.095 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-09-21T23:01:29.097 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-09-21T23:01:29.100 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-09-21T23:01:29.105 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T23:01:29.111 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T23:01:29.111 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-09-21T23:01:29.144 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-09-21T23:01:29.147 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-09-21T23:01:29.167 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T23:01:29.243 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T23:01:29.256 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T23:01:29.257 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T23:01:29.257 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T23:01:29.258 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T23:01:29.258 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T23:01:29.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:01:29.263 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-21T23:01:29.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:01:29.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:01:29.264 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-21T23:01:29.265 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-21T23:01:29.309 INFO:teuthology.orchestra.run.smithi144.stdout:5.15.0-56-generic 2024-09-21T23:01:29.309 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.15.0-56-generic 2024-09-21T23:01:29.309 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-09-21T23:01:29.311 INFO:teuthology.orchestra.run.smithi053.stdout:5.15.0-56-generic 2024-09-21T23:01:29.311 INFO:teuthology.task.kernel:Running kernel on smithi053: 5.15.0-56-generic 2024-09-21T23:01:29.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get clean 2024-09-21T23:01:29.313 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-56-generic 2024-09-21T23:01:29.313 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.15.0-56-generic 2024-09-21T23:01:29.313 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-09-21T23:01:29.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get update 2024-09-21T23:01:29.453 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-09-21T23:01:29.519 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T23:01:29.595 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T23:01:29.664 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T23:01:29.712 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:01:29.736 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:01:29.736 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T23:01:29.769 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:01:29.790 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:01:29.793 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:01:29.807 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T23:01:29.851 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:01:29.866 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T23:01:29.910 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T23:01:29.916 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T23:01:29.919 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T23:01:29.919 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T23:01:29.920 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T23:01:29.943 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T23:01:29.947 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T23:01:29.947 INFO:teuthology.orchestra.run.smithi053.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T23:01:29.955 INFO:teuthology.orchestra.run.smithi053.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T23:01:29.961 INFO:teuthology.orchestra.run.smithi053.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T23:01:29.962 INFO:teuthology.orchestra.run.smithi053.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T23:01:29.962 INFO:teuthology.orchestra.run.smithi053.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T23:01:29.963 INFO:teuthology.orchestra.run.smithi053.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T23:01:29.963 INFO:teuthology.orchestra.run.smithi053.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T23:01:29.969 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T23:01:29.970 INFO:teuthology.orchestra.run.smithi053.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T23:01:29.971 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T23:01:29.974 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T23:01:29.974 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T23:01:29.994 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T23:01:29.994 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T23:01:29.998 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T23:01:29.998 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T23:01:30.003 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T23:01:30.009 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T23:01:30.010 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T23:01:30.010 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T23:01:30.011 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T23:01:30.025 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T23:01:30.048 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T23:01:30.145 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T23:01:30.148 INFO:teuthology.orchestra.run.smithi053.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T23:01:30.167 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T23:01:30.205 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T23:01:30.283 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T23:01:30.286 INFO:teuthology.orchestra.run.smithi053.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T23:01:30.343 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T23:01:30.462 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T23:01:30.537 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T23:01:30.604 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T23:01:30.606 INFO:teuthology.orchestra.run.smithi053.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T23:01:30.636 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T23:01:30.638 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T23:01:30.640 INFO:teuthology.orchestra.run.smithi053.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T23:01:30.649 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T23:01:30.650 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T23:01:30.653 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T23:01:30.654 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T23:01:30.654 INFO:teuthology.orchestra.run.smithi053.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T23:01:30.655 INFO:teuthology.orchestra.run.smithi053.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T23:01:30.655 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T23:01:30.697 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T23:01:30.711 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T23:01:30.712 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T23:01:30.714 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T23:01:30.724 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T23:01:30.724 INFO:teuthology.orchestra.run.smithi053.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T23:01:30.725 INFO:teuthology.orchestra.run.smithi053.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T23:01:30.735 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T23:01:30.736 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T23:01:30.741 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T23:01:30.741 INFO:teuthology.orchestra.run.smithi053.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T23:01:30.742 INFO:teuthology.orchestra.run.smithi053.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T23:01:30.764 INFO:teuthology.orchestra.run.smithi053.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T23:01:30.772 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T23:01:30.772 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T23:01:30.773 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T23:01:30.783 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T23:01:30.789 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T23:01:30.793 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T23:01:30.794 INFO:teuthology.orchestra.run.smithi053.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T23:01:30.799 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T23:01:30.801 INFO:teuthology.orchestra.run.smithi053.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T23:01:30.802 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T23:01:30.802 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T23:01:30.802 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T23:01:30.803 INFO:teuthology.orchestra.run.smithi053.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T23:01:30.803 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T23:01:30.803 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T23:01:30.804 INFO:teuthology.orchestra.run.smithi053.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T23:01:30.807 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T23:01:30.811 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T23:01:30.812 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T23:01:30.812 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T23:01:30.813 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T23:01:30.834 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T23:01:30.850 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T23:01:30.850 INFO:teuthology.orchestra.run.smithi053.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T23:01:30.851 INFO:teuthology.orchestra.run.smithi053.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T23:01:30.851 INFO:teuthology.orchestra.run.smithi053.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T23:01:30.851 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T23:01:30.853 INFO:teuthology.orchestra.run.smithi053.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T23:01:30.854 INFO:teuthology.orchestra.run.smithi053.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T23:01:30.857 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T23:01:30.861 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T23:01:30.862 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T23:01:30.863 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T23:01:30.884 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T23:01:30.884 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T23:01:30.895 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T23:01:30.896 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T23:01:30.898 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T23:01:30.898 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T23:01:30.899 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T23:01:30.931 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T23:01:30.932 INFO:teuthology.orchestra.run.smithi053.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T23:01:30.932 INFO:teuthology.orchestra.run.smithi053.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T23:01:30.932 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T23:01:30.933 INFO:teuthology.orchestra.run.smithi053.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T23:01:30.933 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T23:01:30.934 INFO:teuthology.orchestra.run.smithi053.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T23:01:30.934 INFO:teuthology.orchestra.run.smithi053.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T23:01:30.939 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T23:01:30.939 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T23:01:30.941 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T23:01:30.942 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T23:01:30.943 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T23:01:30.943 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T23:01:30.944 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T23:01:31.024 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T23:01:31.024 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T23:01:36.435 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 16.3 MB in 2s (7,502 kB/s) 2024-09-21T23:01:36.686 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 16.3 MB in 2s (7,267 kB/s) 2024-09-21T23:01:36.874 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 16.3 MB in 3s (5,799 kB/s) 2024-09-21T23:01:37.855 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T23:01:37.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:01:37.978 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T23:01:38.189 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T23:01:38.190 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T23:01:38.241 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-21T23:01:38.259 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:01:38.306 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-09-21T23:01:38.313 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-21T23:01:38.325 DEBUG:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:01:38.379 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:01:38.379 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:01:38.380 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:01:38.380 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:01:38.380 INFO:teuthology.orchestra.run.smithi170.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T23:01:38.380 INFO:teuthology.orchestra.run.smithi170.stdout: python2.7-minimal 2024-09-21T23:01:38.380 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:01:38.382 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-09-21T23:01:38.382 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:01:38.382 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T23:01:38.383 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T23:01:38.384 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-09-21T23:01:38.384 INFO:teuthology.orchestra.run.smithi170.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T23:01:38.384 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-09-21T23:01:38.384 INFO:teuthology.orchestra.run.smithi170.stdout: thermald 2024-09-21T23:01:38.389 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-09-21T23:01:38.442 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-09-21T23:01:38.442 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:01:38.442 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T23:01:38.442 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T23:01:38.444 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2024-09-21T23:01:38.444 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T23:01:38.500 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-09-21T23:01:38.501 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-09-21T23:01:38.514 INFO:teuthology.orchestra.run.smithi170.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:01:38.514 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 113 MB of archives. 2024-09-21T23:01:38.515 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T23:01:38.515 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T23:01:38.607 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2024-09-21T23:01:38.608 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2024-09-21T23:01:38.672 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:01:38.672 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:01:38.672 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:01:38.673 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:01:38.673 INFO:teuthology.orchestra.run.smithi144.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T23:01:38.673 INFO:teuthology.orchestra.run.smithi144.stdout: python2.7-minimal 2024-09-21T23:01:38.673 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:01:38.674 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-09-21T23:01:38.674 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:01:38.674 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T23:01:38.675 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T23:01:38.676 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-09-21T23:01:38.676 INFO:teuthology.orchestra.run.smithi144.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T23:01:38.676 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-09-21T23:01:38.676 INFO:teuthology.orchestra.run.smithi144.stdout: thermald 2024-09-21T23:01:38.725 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-09-21T23:01:38.725 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:01:38.725 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T23:01:38.725 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T23:01:38.726 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages will be upgraded: 2024-09-21T23:01:38.727 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T23:01:38.807 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:01:38.807 INFO:teuthology.orchestra.run.smithi053.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:01:38.808 INFO:teuthology.orchestra.run.smithi053.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:01:38.808 INFO:teuthology.orchestra.run.smithi053.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:01:38.808 INFO:teuthology.orchestra.run.smithi053.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T23:01:38.808 INFO:teuthology.orchestra.run.smithi053.stdout: python2.7-minimal 2024-09-21T23:01:38.809 INFO:teuthology.orchestra.run.smithi053.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:01:38.810 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2024-09-21T23:01:38.810 INFO:teuthology.orchestra.run.smithi053.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:01:38.810 INFO:teuthology.orchestra.run.smithi053.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T23:01:38.811 INFO:teuthology.orchestra.run.smithi053.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T23:01:38.812 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2024-09-21T23:01:38.812 INFO:teuthology.orchestra.run.smithi053.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T23:01:38.812 INFO:teuthology.orchestra.run.smithi053.stdout:Recommended packages: 2024-09-21T23:01:38.812 INFO:teuthology.orchestra.run.smithi053.stdout: thermald 2024-09-21T23:01:38.869 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2024-09-21T23:01:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T23:01:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T23:01:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T23:01:38.872 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages will be upgraded: 2024-09-21T23:01:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T23:01:38.906 INFO:teuthology.orchestra.run.smithi144.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:01:38.906 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 113 MB of archives. 2024-09-21T23:01:38.906 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T23:01:38.907 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T23:01:38.924 INFO:teuthology.orchestra.run.smithi053.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:01:38.924 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 113 MB of archives. 2024-09-21T23:01:38.925 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T23:01:38.925 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T23:01:38.962 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T23:01:39.065 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T23:01:39.366 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T23:01:39.477 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T23:01:39.914 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T23:01:39.914 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T23:01:39.915 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T23:01:40.041 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T23:01:40.063 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T23:01:40.293 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T23:01:40.294 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T23:01:40.294 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T23:01:40.327 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T23:01:40.425 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T23:01:40.445 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 113 MB in 2s (70.3 MB/s) 2024-09-21T23:01:40.524 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T23:01:40.616 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T23:01:40.676 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T23:01:40.946 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 113 MB in 2s (68.9 MB/s) 2024-09-21T23:01:41.126 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T23:01:42.762 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T23:01:42.766 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:42.871 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T23:01:42.871 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T23:01:42.871 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T23:01:42.897 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:43.170 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T23:01:43.174 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:43.217 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T23:01:43.290 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:43.297 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T23:01:43.728 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 113 MB in 5s (24.8 MB/s) 2024-09-21T23:01:43.907 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T23:01:44.965 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T23:01:44.986 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:45.063 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:45.535 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T23:01:45.555 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:45.593 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:45.626 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T23:01:45.648 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:45.745 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:46.041 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T23:01:46.047 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:46.180 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:46.301 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T23:01:46.321 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:46.574 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:48.630 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T23:01:48.651 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:48.748 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:49.158 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T23:01:49.178 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:49.208 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:52.204 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:01:52.319 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:01:52.678 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:01:52.806 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:01:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T23:01:53.165 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T23:01:53.209 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:01:53.384 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:01:53.494 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:01:53.860 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:01:53.988 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:01:54.308 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T23:01:54.331 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T23:01:54.375 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:01:55.560 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:01:55.678 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:01:55.978 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:01:56.098 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:01:56.434 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T23:01:56.446 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T23:01:56.485 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:01:58.596 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T23:01:58.635 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:01:59.012 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:01:59.973 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T23:02:00.011 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:02:00.056 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:01.610 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:02:01.724 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:02:02.032 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:02:02.086 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T23:02:02.123 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T23:02:02.160 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:02.171 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:02.297 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T23:02:02.426 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:03.299 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:02:03.424 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:02:03.784 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:03.784 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:03.784 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:03.784 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:03.800 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:02:03.936 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:03.937 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:04.080 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T23:02:04.208 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:04.723 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T23:02:04.869 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T23:02:05.636 INFO:teuthology.orchestra.run.smithi053.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:05.637 INFO:teuthology.orchestra.run.smithi053.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:05.637 INFO:teuthology.orchestra.run.smithi053.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:05.637 INFO:teuthology.orchestra.run.smithi053.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:05.641 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T23:02:05.938 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T23:02:06.063 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T23:02:06.070 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:06.189 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:06.279 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:06.405 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T23:02:06.526 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:07.529 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T23:02:07.616 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:07.655 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T23:02:07.677 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T23:02:07.677 INFO:teuthology.orchestra.run.smithi170.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:07.790 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:07.838 INFO:teuthology.orchestra.run.smithi144.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:07.838 INFO:teuthology.orchestra.run.smithi144.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:07.838 INFO:teuthology.orchestra.run.smithi144.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:07.838 INFO:teuthology.orchestra.run.smithi144.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:07.985 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:09.233 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T23:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:09.370 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T23:02:09.513 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T23:02:09.639 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:10.990 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T23:02:11.048 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T23:02:11.048 INFO:teuthology.orchestra.run.smithi144.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:22.489 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T23:02:22.490 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub' 2024-09-21T23:02:22.503 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:02:22.558 INFO:teuthology.orchestra.run.smithi170.stdout:Generating grub configuration file ... 2024-09-21T23:02:22.860 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:22.877 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:23.245 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:23.251 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:23.326 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:02:23.332 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:02:23.494 INFO:teuthology.orchestra.run.smithi170.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:02:23.494 INFO:teuthology.orchestra.run.smithi170.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:02:23.494 INFO:teuthology.orchestra.run.smithi170.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:02:23.533 INFO:teuthology.orchestra.run.smithi170.stdout:done 2024-09-21T23:02:23.980 INFO:teuthology.orchestra.run.smithi053.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T23:02:23.981 INFO:teuthology.orchestra.run.smithi053.stdout:Sourcing file `/etc/default/grub' 2024-09-21T23:02:23.998 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-09-21T23:02:24.002 INFO:teuthology.orchestra.run.smithi053.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:02:24.017 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-09-21T23:02:24.017 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-09-21T23:02:24.017 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.122.122 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:02:24.018 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-09-21T23:02:24.019 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T23:02:24.019 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T23:02:24.019 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T23:02:24.019 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T23:02:24.019 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T23:02:24.019 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-09-21T23:02:24.028 INFO:teuthology.orchestra.run.smithi053.stdout:Generating grub configuration file ... 2024-09-21T23:02:24.200 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T23:02:24.317 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:02:24.325 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:24.327 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:02:24.343 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:24.345 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:02:24.439 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:02:24.692 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:24.698 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:24.764 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:02:24.770 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:02:24.915 INFO:teuthology.orchestra.run.smithi053.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:02:24.915 INFO:teuthology.orchestra.run.smithi053.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:02:24.915 INFO:teuthology.orchestra.run.smithi053.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:02:24.962 INFO:teuthology.orchestra.run.smithi053.stdout:done 2024-09-21T23:02:25.178 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T23:02:25.178 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub' 2024-09-21T23:02:25.189 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:02:25.198 INFO:teuthology.orchestra.run.smithi144.stdout:Generating grub configuration file ... 2024-09-21T23:02:25.464 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:25.477 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:25.487 DEBUG:teuthology.orchestra.run.smithi053:> dpkg -s linux-image-generic 2024-09-21T23:02:25.549 INFO:teuthology.orchestra.run.smithi053.stdout:Package: linux-image-generic 2024-09-21T23:02:25.549 INFO:teuthology.orchestra.run.smithi053.stdout:Status: install ok installed 2024-09-21T23:02:25.549 INFO:teuthology.orchestra.run.smithi053.stdout:Priority: optional 2024-09-21T23:02:25.549 INFO:teuthology.orchestra.run.smithi053.stdout:Section: kernel 2024-09-21T23:02:25.549 INFO:teuthology.orchestra.run.smithi053.stdout:Installed-Size: 21 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: amd64 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Source: linux-meta 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Version: 5.15.0.122.122 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Recommends: thermald 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:Description: Generic Linux kernel image 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:02:25.550 INFO:teuthology.orchestra.run.smithi053.stdout: available. 2024-09-21T23:02:25.550 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T23:02:25.551 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T23:02:25.551 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T23:02:25.551 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T23:02:25.551 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T23:02:25.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get clean 2024-09-21T23:02:25.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get update 2024-09-21T23:02:25.750 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:25.755 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:25.764 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T23:02:25.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:02:25.824 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:02:25.830 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:02:25.837 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:02:25.837 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:02:25.843 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:02:25.846 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T23:02:25.860 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:02:25.991 INFO:teuthology.orchestra.run.smithi144.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:02:25.991 INFO:teuthology.orchestra.run.smithi144.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:02:25.991 INFO:teuthology.orchestra.run.smithi144.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:02:26.033 INFO:teuthology.orchestra.run.smithi144.stdout:done 2024-09-21T23:02:26.058 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T23:02:26.059 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T23:02:26.255 INFO:teuthology.orchestra.run.smithi170.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T23:02:26.256 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:02:26.256 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:02:26.256 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:02:26.256 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:02:26.256 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:02:26.257 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:02:26.257 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:02:26.300 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:02:26.302 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-09-21T23:02:26.318 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.122.122 2024-09-21T23:02:26.319 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:02:26.319 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:02:26.319 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-09-21T23:02:26.319 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-09-21T23:02:26.319 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:02:26.319 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-09-21T23:02:26.319 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-09-21T23:02:26.362 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.jZlARi3iYJ 2024-09-21T23:02:26.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub/grub.cfg /tmp/tmp.jZlARi3iYJ 2024-09-21T23:02:26.416 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.jZlARi3iYJ 2024-09-21T23:02:26.537 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.jZlARi3iYJ is 10KB 2024-09-21T23:02:26.546 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-09-21T23:02:26.586 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.jZlARi3iYJ 2024-09-21T23:02:26.590 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T23:02:26.599 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 21 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.15.0.122.122 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:02:26.600 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-09-21T23:02:26.601 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-09-21T23:02:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:02:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-09-21T23:02:26.601 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T23:02:26.601 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T23:02:26.601 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T23:02:26.601 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T23:02:26.601 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T23:02:26.601 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-09-21T23:02:26.642 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:02:26.642 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T23:02:26.642 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T23:02:26.716 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T23:02:26.717 DEBUG:teuthology.orchestra.run.smithi170:> sudo update-grub 2024-09-21T23:02:26.795 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-09-21T23:02:26.927 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:02:26.928 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:02:26.936 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:02:26.954 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:02:27.321 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-09-21T23:02:27.340 DEBUG:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:02:27.406 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-09-21T23:02:27.623 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2024-09-21T23:02:27.623 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2024-09-21T23:02:27.824 INFO:teuthology.orchestra.run.smithi053.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T23:02:27.824 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:02:27.824 INFO:teuthology.orchestra.run.smithi053.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:02:27.825 INFO:teuthology.orchestra.run.smithi053.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:02:27.825 INFO:teuthology.orchestra.run.smithi053.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:02:27.825 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:02:27.825 INFO:teuthology.orchestra.run.smithi053.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:02:27.825 INFO:teuthology.orchestra.run.smithi053.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:02:27.872 INFO:teuthology.orchestra.run.smithi053.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:02:27.874 DEBUG:teuthology.orchestra.run.smithi053:> dpkg -s linux-image-generic 2024-09-21T23:02:27.890 INFO:teuthology.orchestra.run.smithi053.stdout:Package: linux-image-generic 2024-09-21T23:02:27.890 INFO:teuthology.orchestra.run.smithi053.stdout:Status: install ok installed 2024-09-21T23:02:27.890 INFO:teuthology.orchestra.run.smithi053.stdout:Priority: optional 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Section: kernel 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Installed-Size: 21 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: amd64 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Source: linux-meta 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Version: 5.15.0.122.122 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Recommends: thermald 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout:Description: Generic Linux kernel image 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:02:27.891 INFO:teuthology.orchestra.run.smithi053.stdout: available. 2024-09-21T23:02:27.892 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-21T23:02:27.939 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.ZFChbefDX1 2024-09-21T23:02:27.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ZFChbefDX1 2024-09-21T23:02:27.994 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.ZFChbefDX1 2024-09-21T23:02:28.014 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub' 2024-09-21T23:02:28.015 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:02:28.018 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-09-21T23:02:28.119 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.ZFChbefDX1 is 10KB 2024-09-21T23:02:28.132 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.ZFChbefDX1 2024-09-21T23:02:28.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T23:02:28.195 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:02:28.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T23:02:28.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T23:02:28.253 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:28.259 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:28.286 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T23:02:28.287 DEBUG:teuthology.orchestra.run.smithi053:> sudo update-grub 2024-09-21T23:02:28.383 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-21T23:02:28.401 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T23:02:28.459 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-21T23:02:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:28.559 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:28.602 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:02:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:02:28.670 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-09-21T23:02:28.671 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-09-21T23:02:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:02:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:02:28.730 INFO:teuthology.orchestra.run.smithi170.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:02:28.741 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-09-21T23:02:28.754 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-09-21T23:02:28.850 INFO:teuthology.orchestra.run.smithi144.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T23:02:28.850 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:02:28.850 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:02:28.850 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:02:28.850 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:02:28.851 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:02:28.851 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:02:28.851 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:02:28.890 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T23:02:28.892 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-09-21T23:02:28.908 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-09-21T23:02:28.908 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-09-21T23:02:28.908 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 21 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.15.0.122.122 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-09-21T23:02:28.909 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-09-21T23:02:28.910 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-09-21T23:02:28.956 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.WJJ4VaH0lb 2024-09-21T23:02:28.957 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub/grub.cfg /tmp/tmp.WJJ4VaH0lb 2024-09-21T23:02:29.011 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.WJJ4VaH0lb 2024-09-21T23:02:29.123 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.WJJ4VaH0lb is 10KB 2024-09-21T23:02:29.135 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.WJJ4VaH0lb 2024-09-21T23:02:29.139 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T23:02:29.191 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:02:29.191 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T23:02:29.191 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T23:02:29.255 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T23:02:29.255 DEBUG:teuthology.orchestra.run.smithi144:> sudo update-grub 2024-09-21T23:02:29.649 INFO:teuthology.orchestra.run.smithi053.stderr:Sourcing file `/etc/default/grub' 2024-09-21T23:02:29.651 INFO:teuthology.orchestra.run.smithi053.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:02:29.653 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-09-21T23:02:29.841 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:29.848 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:30.115 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:30.158 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:02:30.164 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:02:30.304 INFO:teuthology.orchestra.run.smithi053.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:02:30.304 INFO:teuthology.orchestra.run.smithi053.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:02:30.304 INFO:teuthology.orchestra.run.smithi053.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:02:30.320 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-09-21T23:02:30.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-09-21T23:02:30.516 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub' 2024-09-21T23:02:30.518 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T23:02:30.519 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-09-21T23:02:30.688 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T23:02:30.694 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T23:02:30.945 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T23:02:30.951 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T23:02:30.991 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T23:02:30.996 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T23:02:31.130 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T23:02:31.131 INFO:teuthology.orchestra.run.smithi144.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T23:02:31.131 INFO:teuthology.orchestra.run.smithi144.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T23:02:31.141 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-09-21T23:02:31.151 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-09-21T23:02:58.782 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T23:02:58.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T23:02:58.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:00.334 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T23:03:00.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T23:03:00.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:01.154 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T23:03:01.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-21T23:03:01.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:17.231 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T23:03:18.767 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-09-21T23:03:19.535 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-09-21T23:03:26.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T23:03:26.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:27.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T23:03:27.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:28.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-21T23:03:28.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:29.295 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T23:03:31.599 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-09-21T23:03:41.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T23:03:41.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:43.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-21T23:03:43.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:46.159 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-09-21T23:03:58.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T23:03:58.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:03:59.727 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T23:04:01.231 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-09-21T23:04:14.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T23:04:14.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:04:15.047 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-21T23:04:15.455 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-21T23:04:15.939 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T23:04:15.939 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.15.0-122-generic"... 2024-09-21T23:04:15.939 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-21T23:04:15.985 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-122-generic 2024-09-21T23:04:15.985 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T23:04:15.985 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T23:04:15.985 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T23:04:16.037 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-21T23:04:16.037 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-21T23:04:16.038 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-21T23:04:16.085 INFO:teuthology.orchestra.run.smithi144.stdout:5.15.0-122-generic 2024-09-21T23:04:16.086 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T23:04:16.086 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T23:04:16.086 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T23:04:16.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T23:04:16.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:04:16.533 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-21T23:04:16.986 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-21T23:04:16.986 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T23:04:17.073 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-09-21T23:04:17.086 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T23:04:17.086 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T23:04:17.094 DEBUG:teuthology.parallel:result is None 2024-09-21T23:04:17.160 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-09-21T23:04:17.179 DEBUG:teuthology.parallel:result is None 2024-09-21T23:04:17.199 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T23:04:17.199 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-21T23:04:17.199 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-21T23:04:17.247 INFO:teuthology.orchestra.run.smithi053.stdout:5.15.0-122-generic 2024-09-21T23:04:17.247 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T23:04:17.247 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T23:04:17.247 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T23:04:18.248 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T23:04:18.248 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T23:04:18.331 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-09-21T23:04:18.360 DEBUG:teuthology.parallel:result is None 2024-09-21T23:04:18.360 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T23:04:18.367 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T23:04:18.367 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T23:04:18.369 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T23:04:18.371 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T23:04:18.379 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T23:04:18.410 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T23:04:18.428 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T23:04:18.428 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T23:04:18.431 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T23:04:18.433 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T23:04:18.459 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T23:04:18.466 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T23:04:18.466 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T23:04:18.484 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T23:04:18.495 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T23:04:18.500 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:04:18.509 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:04:18.516 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:04:18.524 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:04:18.524 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:04:18.533 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T23:04:18.535 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T23:04:18.541 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T23:04:18.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T23:04:18.555 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T23:04:18.574 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T23:04:18.589 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T23:04:18.597 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T23:04:18.597 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T23:04:18.612 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T23:04:18.627 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T23:04:18.636 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T23:04:18.663 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T23:04:18.711 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:04:18.711 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T23:04:18.767 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T23:04:18.772 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T23:04:18.818 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:04:18.818 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T23:04:18.881 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T23:04:18.887 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T23:04:18.933 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:04:18.933 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T23:04:18.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-09-21T23:04:18.995 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-09-21T23:04:18.997 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-09-21T23:04:19.067 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T23:04:19.075 INFO:teuthology.task.internal:Starting timer... 2024-09-21T23:04:19.075 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T23:04:19.100 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T23:04:19.108 DEBUG:teuthology.task.selinux:Excluding smithi053: OS 'ubuntu' does not support SELinux 2024-09-21T23:04:19.108 DEBUG:teuthology.task.selinux:Excluding smithi144: OS 'ubuntu' does not support SELinux 2024-09-21T23:04:19.108 DEBUG:teuthology.task.selinux:Excluding smithi170: OS 'ubuntu' does not support SELinux 2024-09-21T23:04:19.108 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T23:04:19.108 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T23:04:19.108 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T23:04:19.109 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T23:04:19.117 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T23:04:19.386 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T23:04:19.431 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi144.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T23:11:54.568 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-09-21T23:11:54.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T23:11:54.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:11:54.649 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-21T23:11:54.726 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T23:11:54.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-21T23:11:54.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:11:54.805 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-21T23:11:54.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T23:11:54.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T23:11:54.961 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-21T23:11:55.034 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T23:11:55.035 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T23:11:55.045 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T23:11:55.045 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:11:55.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:11:55.048 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:11:55.048 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:11:55.050 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T23:11:55.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:11:55.078 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:11:55.079 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Command line: ntpd -gq 2024-09-21T23:11:55.079 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: ---------------------------------------------------- 2024-09-21T23:11:55.079 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:11:55.079 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:11:55.079 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: corporation. Support and training for ntp-4 are 2024-09-21T23:11:55.079 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: available at https://www.nwtime.org/support 2024-09-21T23:11:55.079 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: ---------------------------------------------------- 2024-09-21T23:11:55.080 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: proto: precision = 0.062 usec (-24) 2024-09-21T23:11:55.080 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: basedate set to 2022-02-04 2024-09-21T23:11:55.080 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:11:55.080 INFO:teuthology.orchestra.run.smithi144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:11:55.080 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:11:55.081 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:11:55.081 INFO:teuthology.orchestra.run.smithi144.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:11:55.082 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:11:55.082 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:11:55.082 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:11:55.082 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Listen normally on 3 enp3s0f1 172.21.15.144:123 2024-09-21T23:11:55.082 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Listen normally on 4 lo [::1]:123 2024-09-21T23:11:55.082 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5f%5]:123 2024-09-21T23:11:55.082 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:11:55 ntpd[18474]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:11:55.083 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:11:55.083 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Command line: ntpd -gq 2024-09-21T23:11:55.083 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: ---------------------------------------------------- 2024-09-21T23:11:55.083 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:11:55.083 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: corporation. Support and training for ntp-4 are 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: available at https://www.nwtime.org/support 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: ---------------------------------------------------- 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: proto: precision = 0.067 usec (-24) 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: basedate set to 2022-02-04 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:11:55.084 INFO:teuthology.orchestra.run.smithi053.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:11:55.085 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Listen normally on 3 ens1f0 172.21.15.53:123 2024-09-21T23:11:55.086 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Listen normally on 4 lo [::1]:123 2024-09-21T23:11:55.086 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:3718%4]:123 2024-09-21T23:11:55.086 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:11:55 ntpd[18368]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:11:55.102 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T23:11:55.102 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Command line: ntpd -gq 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: ---------------------------------------------------- 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: corporation. Support and training for ntp-4 are 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: available at https://www.nwtime.org/support 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: ---------------------------------------------------- 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: proto: precision = 0.044 usec (-24) 2024-09-21T23:11:55.103 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: basedate set to 2022-02-04 2024-09-21T23:11:55.104 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: gps base set to 2022-02-06 (week 2196) 2024-09-21T23:11:55.104 INFO:teuthology.orchestra.run.smithi170.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:11:55.104 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T23:11:55.104 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:11:55.104 INFO:teuthology.orchestra.run.smithi170.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T23:11:55.105 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T23:11:55.105 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T23:11:55.105 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T23:11:55.105 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Listen normally on 3 enp3s0f1 172.21.15.170:123 2024-09-21T23:11:55.106 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Listen normally on 4 lo [::1]:123 2024-09-21T23:11:55.106 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7255%5]:123 2024-09-21T23:11:55.106 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:11:55 ntpd[18459]: Listening on routing socket on fd #22 for interface updates 2024-09-21T23:11:56.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:56 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:56.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:56 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:56.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:56 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:56.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:56 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:56.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:56 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:56.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:56 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:57.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:57 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:57.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:57 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:57.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:57 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:57.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:57 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:57.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:57 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:57.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:57 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:57.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:57.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:57.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:57.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:57.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:57.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:57.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:57.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:57 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:57.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:57 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:57.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:57 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:58.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:58 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:58.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:58 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:58.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:58 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:58.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:58 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:58.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:58 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:58.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:58 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:58 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:58 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:58 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:58 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:58 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:58 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.081 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:59.081 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:11:59 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.083 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.083 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:59.083 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:11:59 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:11:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:59 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:59 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:59 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:11:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:59 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:11:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:59 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:11:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:11:59 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:00.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:00.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:00 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:00.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:00 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:00 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:00 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:00 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:00 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:00.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:00 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:00 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:01.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:01.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:01 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:01 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:01.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:01 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:01 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:01 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:01.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:01 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:01.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:01 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:01.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:01 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:02.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.079 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:02.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:02.080 INFO:teuthology.orchestra.run.smithi144.stdout:21 Sep 23:12:02 ntpd[18474]: ntpd: time slew +0.000596 s 2024-09-21T23:12:02.080 INFO:teuthology.orchestra.run.smithi144.stdout:ntpd: time slew +0.000596s 2024-09-21T23:12:02.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:12:02.080 INFO:teuthology.orchestra.run.smithi144.stderr:21 Sep 23:12:02 ntpd[18474]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:12:02.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:02 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:02 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:02 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:02 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.081 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:02 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:02 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:02.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:02.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:02 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:02.139 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:12:02.140 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-09-21T23:12:02.140 INFO:teuthology.orchestra.run.smithi144.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:02.140 INFO:teuthology.orchestra.run.smithi144.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:02.140 INFO:teuthology.orchestra.run.smithi144.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:02.140 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:03.082 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi053.stdout:21 Sep 23:12:03 ntpd[18368]: ntpd: time slew -0.002315 s 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi053.stdout:ntpd: time slew -0.002315s 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:12:03.083 INFO:teuthology.orchestra.run.smithi053.stderr:21 Sep 23:12:03 ntpd[18368]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:03 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:03 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:03 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:03 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:03 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:03.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:03 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================== 2024-09-21T23:12:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:04.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:04.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:04 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T23:12:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T23:12:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T23:12:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T23:12:05.104 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 23:12:05 ntpd[18459]: ntpd: time slew -0.003302 s 2024-09-21T23:12:05.104 INFO:teuthology.orchestra.run.smithi170.stdout:ntpd: time slew -0.003302s 2024-09-21T23:12:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T23:12:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 23:12:05 ntpd[18459]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T23:12:05.157 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:12:05.157 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-09-21T23:12:05.157 INFO:teuthology.orchestra.run.smithi170.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:05.157 INFO:teuthology.orchestra.run.smithi170.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:05.157 INFO:teuthology.orchestra.run.smithi170.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:05.157 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:12:05.158 INFO:teuthology.run_tasks:Running task install... 2024-09-21T23:12:05.169 DEBUG:teuthology.task.install:project ceph 2024-09-21T23:12:05.170 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T23:12:05.170 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-21T23:12:05.170 INFO:teuthology.task.install:Using flavor: default 2024-09-21T23:12:05.175 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T23:12:05.175 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-21T23:12:05.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-key list | grep Ceph 2024-09-21T23:12:05.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-key list | grep Ceph 2024-09-21T23:12:05.176 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-key list | grep Ceph 2024-09-21T23:12:05.241 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:12:05.242 INFO:teuthology.orchestra.run.smithi053.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:12:05.257 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T23:12:05.271 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:12:05.271 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:12:05.272 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-21T23:12:05.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:12:05.276 INFO:teuthology.orchestra.run.smithi053.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:12:05.276 INFO:teuthology.orchestra.run.smithi053.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:12:05.277 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-21T23:12:05.277 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:12:05.285 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T23:12:05.285 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T23:12:05.285 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-21T23:12:05.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:12:05.416 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:12:05.416 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:12:05.420 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:12:05.420 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:12:05.430 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-21T23:12:05.430 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:12:05.528 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:12:05.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:12:05.536 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:12:05.536 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:12:05.538 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:12:05.538 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T23:12:05.542 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-09-21T23:12:05.552 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T23:12:05.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get update 2024-09-21T23:12:05.710 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:12:05.720 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T23:12:05.737 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:12:05.766 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:12:05.778 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:12:05.780 INFO:teuthology.orchestra.run.smithi053.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T23:12:05.786 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:12:05.787 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:12:05.796 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T23:12:05.800 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:12:05.801 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T23:12:05.809 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:12:05.822 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:12:05.825 INFO:teuthology.orchestra.run.smithi053.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T23:12:05.830 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:12:05.843 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:12:05.846 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:12:05.850 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:12:05.856 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T23:12:05.857 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:12:05.874 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-21T23:12:05.885 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-21T23:12:05.916 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T23:12:05.928 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:12:05.929 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T23:12:06.069 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 28.9 kB in 0s (83.9 kB/s) 2024-09-21T23:12:06.283 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:12:06.285 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T23:12:06.445 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 157 kB in 1s (223 kB/s) 2024-09-21T23:12:06.532 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 286 kB in 1s (362 kB/s) 2024-09-21T23:12:07.079 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-21T23:12:07.096 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy rbd-nbd=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:12:07.149 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-21T23:12:07.340 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-09-21T23:12:07.341 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-09-21T23:12:07.432 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T23:12:07.448 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy rbd-nbd=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:12:07.514 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T23:12:07.533 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:12:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:12:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:12:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:12:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:12:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:12:07.534 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:12:07.535 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-09-21T23:12:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:12:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:12:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:12:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:12:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:12:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:12:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:12:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:12:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet zip 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:12:07.538 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-09-21T23:12:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: btrfs-tools 2024-09-21T23:12:07.566 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-09-21T23:12:07.584 DEBUG:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy rbd-nbd=19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:12:07.641 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-09-21T23:12:07.641 INFO:teuthology.orchestra.run.smithi144.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:12:07.641 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:12:07.641 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi144.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:12:07.642 INFO:teuthology.orchestra.run.smithi144.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:12:07.643 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:12:07.644 INFO:teuthology.orchestra.run.smithi144.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-21T23:12:07.648 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-09-21T23:12:07.725 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T23:12:07.726 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T23:12:07.786 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 119 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:12:07.786 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 181 MB of archives. 2024-09-21T23:12:07.786 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:12:07.786 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:12:07.831 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:12:07.862 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2024-09-21T23:12:07.863 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2024-09-21T23:12:07.937 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:12:07.938 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:12:07.938 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:12:07.938 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:12:07.938 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:12:07.938 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:12:07.938 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:12:07.939 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-09-21T23:12:07.939 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:12:07.940 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:12:07.940 INFO:teuthology.orchestra.run.smithi170.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:12:07.940 INFO:teuthology.orchestra.run.smithi170.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:12:07.941 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet zip 2024-09-21T23:12:07.942 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-09-21T23:12:07.943 INFO:teuthology.orchestra.run.smithi170.stdout: btrfs-tools 2024-09-21T23:12:07.979 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:12:08.046 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:12:08.047 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-09-21T23:12:08.047 INFO:teuthology.orchestra.run.smithi170.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:12:08.047 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:12:08.047 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:12:08.048 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:12:08.048 INFO:teuthology.orchestra.run.smithi170.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:12:08.048 INFO:teuthology.orchestra.run.smithi170.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:12:08.049 INFO:teuthology.orchestra.run.smithi170.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:12:08.049 INFO:teuthology.orchestra.run.smithi170.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:12:08.049 INFO:teuthology.orchestra.run.smithi170.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:12:08.049 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:12:08.049 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:12:08.050 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:12:08.051 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:12:08.051 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:12:08.051 INFO:teuthology.orchestra.run.smithi170.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-21T23:12:08.057 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:12:08.065 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:12:08.067 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:12:08.069 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:12:08.070 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:12:08.076 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:12:08.084 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T23:12:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T23:12:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T23:12:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T23:12:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T23:12:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T23:12:08.085 INFO:teuthology.orchestra.run.smithi053.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T23:12:08.086 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2024-09-21T23:12:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T23:12:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T23:12:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-21T23:12:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:12:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-21T23:12:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet zip 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout:Recommended packages: 2024-09-21T23:12:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: btrfs-tools 2024-09-21T23:12:08.115 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 119 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:12:08.115 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 181 MB of archives. 2024-09-21T23:12:08.115 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:12:08.115 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:12:08.184 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:12:08.186 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:12:08.193 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:12:08.194 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:12:08.200 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2024-09-21T23:12:08.200 INFO:teuthology.orchestra.run.smithi053.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T23:12:08.200 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T23:12:08.200 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T23:12:08.200 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-21T23:12:08.200 INFO:teuthology.orchestra.run.smithi053.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T23:12:08.200 INFO:teuthology.orchestra.run.smithi053.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-21T23:12:08.201 INFO:teuthology.orchestra.run.smithi053.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T23:12:08.202 INFO:teuthology.orchestra.run.smithi053.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-21T23:12:08.203 INFO:teuthology.orchestra.run.smithi053.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-21T23:12:08.204 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:12:08.206 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:12:08.223 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:12:08.252 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:12:08.263 INFO:teuthology.orchestra.run.smithi053.stdout:0 upgraded, 119 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T23:12:08.263 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 181 MB of archives. 2024-09-21T23:12:08.263 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-21T23:12:08.263 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T23:12:08.264 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:12:08.282 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:12:08.287 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:12:08.299 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:12:08.303 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:12:08.304 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:12:08.305 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:12:08.313 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:12:08.315 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:12:08.318 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:12:08.319 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:12:08.321 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:12:08.323 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:12:08.337 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:12:08.339 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:12:08.340 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:12:08.340 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:12:08.341 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:12:08.341 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:12:08.342 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:12:08.342 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:12:08.345 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:12:08.350 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T23:12:08.353 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T23:12:08.355 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:12:08.355 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:12:08.370 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T23:12:08.372 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:12:08.375 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:12:08.375 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:12:08.376 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:12:08.376 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:12:08.377 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:12:08.378 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:12:08.378 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:12:08.389 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:12:08.390 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:12:08.403 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-21T23:12:08.406 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:12:08.412 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:12:08.413 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:12:08.415 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:12:08.417 INFO:teuthology.orchestra.run.smithi170.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:12:08.423 INFO:teuthology.orchestra.run.smithi170.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:12:08.424 INFO:teuthology.orchestra.run.smithi170.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:12:08.431 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:12:08.437 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:12:08.440 INFO:teuthology.orchestra.run.smithi170.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:12:08.441 INFO:teuthology.orchestra.run.smithi170.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:12:08.442 INFO:teuthology.orchestra.run.smithi170.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:12:08.445 INFO:teuthology.orchestra.run.smithi170.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:12:08.449 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:12:08.453 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:12:08.454 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:12:08.455 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:12:08.463 INFO:teuthology.orchestra.run.smithi053.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:12:08.465 INFO:teuthology.orchestra.run.smithi053.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:12:08.468 INFO:teuthology.orchestra.run.smithi053.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:12:08.469 INFO:teuthology.orchestra.run.smithi053.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:12:08.470 INFO:teuthology.orchestra.run.smithi053.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:12:08.472 INFO:teuthology.orchestra.run.smithi053.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:12:08.487 INFO:teuthology.orchestra.run.smithi053.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:12:08.488 INFO:teuthology.orchestra.run.smithi053.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:12:08.489 INFO:teuthology.orchestra.run.smithi053.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:12:08.489 INFO:teuthology.orchestra.run.smithi053.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:12:08.491 INFO:teuthology.orchestra.run.smithi053.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:12:08.491 INFO:teuthology.orchestra.run.smithi053.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:12:08.498 INFO:teuthology.orchestra.run.smithi170.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:12:08.499 INFO:teuthology.orchestra.run.smithi170.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:12:08.501 INFO:teuthology.orchestra.run.smithi170.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:12:08.501 INFO:teuthology.orchestra.run.smithi170.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:12:08.502 INFO:teuthology.orchestra.run.smithi170.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:12:08.504 INFO:teuthology.orchestra.run.smithi053.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:12:08.504 INFO:teuthology.orchestra.run.smithi053.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:12:08.504 INFO:teuthology.orchestra.run.smithi053.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:12:08.505 INFO:teuthology.orchestra.run.smithi053.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:12:08.519 INFO:teuthology.orchestra.run.smithi170.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:12:08.521 INFO:teuthology.orchestra.run.smithi170.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:12:08.521 INFO:teuthology.orchestra.run.smithi170.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:12:08.523 INFO:teuthology.orchestra.run.smithi053.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:12:08.526 INFO:teuthology.orchestra.run.smithi053.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:12:08.526 INFO:teuthology.orchestra.run.smithi053.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:12:08.527 INFO:teuthology.orchestra.run.smithi053.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:12:08.527 INFO:teuthology.orchestra.run.smithi053.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:12:08.528 INFO:teuthology.orchestra.run.smithi053.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:12:08.536 INFO:teuthology.orchestra.run.smithi170.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:12:08.536 INFO:teuthology.orchestra.run.smithi170.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:12:08.539 INFO:teuthology.orchestra.run.smithi170.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:12:08.540 INFO:teuthology.orchestra.run.smithi170.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:12:08.540 INFO:teuthology.orchestra.run.smithi170.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:12:08.540 INFO:teuthology.orchestra.run.smithi170.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:12:08.542 INFO:teuthology.orchestra.run.smithi053.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:12:08.542 INFO:teuthology.orchestra.run.smithi053.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:12:08.544 INFO:teuthology.orchestra.run.smithi053.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:12:08.544 INFO:teuthology.orchestra.run.smithi053.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:12:08.557 INFO:teuthology.orchestra.run.smithi170.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:12:08.557 INFO:teuthology.orchestra.run.smithi170.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:12:08.559 INFO:teuthology.orchestra.run.smithi053.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:12:08.560 INFO:teuthology.orchestra.run.smithi170.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:12:08.562 INFO:teuthology.orchestra.run.smithi170.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:12:08.563 INFO:teuthology.orchestra.run.smithi170.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:12:08.564 INFO:teuthology.orchestra.run.smithi170.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:12:08.565 INFO:teuthology.orchestra.run.smithi053.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:12:08.565 INFO:teuthology.orchestra.run.smithi053.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:12:08.565 INFO:teuthology.orchestra.run.smithi053.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:12:08.566 INFO:teuthology.orchestra.run.smithi053.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:12:08.568 INFO:teuthology.orchestra.run.smithi053.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:12:08.574 INFO:teuthology.orchestra.run.smithi170.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:12:08.575 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T23:12:08.576 INFO:teuthology.orchestra.run.smithi170.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:12:08.577 INFO:teuthology.orchestra.run.smithi053.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:12:08.577 INFO:teuthology.orchestra.run.smithi170.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:12:08.577 INFO:teuthology.orchestra.run.smithi170.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:12:08.578 INFO:teuthology.orchestra.run.smithi053.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:12:08.579 INFO:teuthology.orchestra.run.smithi053.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:12:08.580 INFO:teuthology.orchestra.run.smithi053.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:12:08.585 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T23:12:08.591 INFO:teuthology.orchestra.run.smithi170.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:12:08.592 INFO:teuthology.orchestra.run.smithi170.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:12:08.594 INFO:teuthology.orchestra.run.smithi053.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:12:08.594 INFO:teuthology.orchestra.run.smithi170.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:12:08.595 INFO:teuthology.orchestra.run.smithi170.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:12:08.595 INFO:teuthology.orchestra.run.smithi053.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:12:08.595 INFO:teuthology.orchestra.run.smithi170.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:12:08.595 INFO:teuthology.orchestra.run.smithi170.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:12:08.596 INFO:teuthology.orchestra.run.smithi053.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:12:08.599 INFO:teuthology.orchestra.run.smithi053.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:12:08.608 INFO:teuthology.orchestra.run.smithi170.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:12:08.609 INFO:teuthology.orchestra.run.smithi170.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:12:08.615 INFO:teuthology.orchestra.run.smithi170.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:12:08.615 INFO:teuthology.orchestra.run.smithi170.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:12:08.620 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T23:12:08.625 INFO:teuthology.orchestra.run.smithi170.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:12:08.627 INFO:teuthology.orchestra.run.smithi170.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:12:08.628 INFO:teuthology.orchestra.run.smithi170.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:12:08.629 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T23:12:08.630 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T23:12:08.632 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T23:12:08.648 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T23:12:08.655 INFO:teuthology.orchestra.run.smithi053.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:12:08.656 INFO:teuthology.orchestra.run.smithi053.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:12:08.658 INFO:teuthology.orchestra.run.smithi053.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:12:08.658 INFO:teuthology.orchestra.run.smithi053.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:12:08.659 INFO:teuthology.orchestra.run.smithi053.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:12:08.677 INFO:teuthology.orchestra.run.smithi053.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:12:08.679 INFO:teuthology.orchestra.run.smithi053.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:12:08.679 INFO:teuthology.orchestra.run.smithi053.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:12:08.694 INFO:teuthology.orchestra.run.smithi053.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:12:08.694 INFO:teuthology.orchestra.run.smithi053.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:12:08.694 INFO:teuthology.orchestra.run.smithi053.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:12:08.695 INFO:teuthology.orchestra.run.smithi053.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:12:08.695 INFO:teuthology.orchestra.run.smithi053.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:12:08.696 INFO:teuthology.orchestra.run.smithi053.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:12:08.708 INFO:teuthology.orchestra.run.smithi053.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:12:08.709 INFO:teuthology.orchestra.run.smithi053.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:12:08.712 INFO:teuthology.orchestra.run.smithi053.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:12:08.719 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T23:12:08.725 INFO:teuthology.orchestra.run.smithi053.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:12:08.727 INFO:teuthology.orchestra.run.smithi053.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:12:08.727 INFO:teuthology.orchestra.run.smithi053.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:12:08.730 INFO:teuthology.orchestra.run.smithi053.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:12:08.733 INFO:teuthology.orchestra.run.smithi053.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:12:08.733 INFO:teuthology.orchestra.run.smithi053.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:12:08.734 INFO:teuthology.orchestra.run.smithi053.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:12:08.742 INFO:teuthology.orchestra.run.smithi053.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:12:08.743 INFO:teuthology.orchestra.run.smithi053.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:12:08.745 INFO:teuthology.orchestra.run.smithi053.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:12:08.759 INFO:teuthology.orchestra.run.smithi053.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:12:08.760 INFO:teuthology.orchestra.run.smithi053.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:12:08.760 INFO:teuthology.orchestra.run.smithi053.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:12:08.761 INFO:teuthology.orchestra.run.smithi053.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:12:08.762 INFO:teuthology.orchestra.run.smithi053.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:12:08.768 INFO:teuthology.orchestra.run.smithi053.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:12:08.768 INFO:teuthology.orchestra.run.smithi053.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:12:08.776 INFO:teuthology.orchestra.run.smithi053.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:12:08.778 INFO:teuthology.orchestra.run.smithi053.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:12:08.779 INFO:teuthology.orchestra.run.smithi053.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:12:08.786 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T23:12:08.786 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:12:08.788 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T23:12:08.789 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T23:12:08.794 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T23:12:08.798 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T23:12:08.801 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T23:12:08.803 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T23:12:08.803 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T23:12:08.820 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T23:12:08.855 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:12:08.857 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:12:08.859 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:12:08.872 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:12:08.888 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T23:12:08.888 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T23:12:08.889 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T23:12:08.890 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T23:12:08.891 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T23:12:08.892 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T23:12:08.897 INFO:teuthology.orchestra.run.smithi144.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T23:12:08.903 INFO:teuthology.orchestra.run.smithi144.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T23:12:08.903 INFO:teuthology.orchestra.run.smithi144.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi144.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:12:08.983 INFO:teuthology.orchestra.run.smithi144.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T23:12:08.985 INFO:teuthology.orchestra.run.smithi144.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T23:12:08.986 INFO:teuthology.orchestra.run.smithi144.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T23:12:08.987 INFO:teuthology.orchestra.run.smithi144.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T23:12:08.989 INFO:teuthology.orchestra.run.smithi144.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T23:12:08.991 INFO:teuthology.orchestra.run.smithi144.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T23:12:08.992 INFO:teuthology.orchestra.run.smithi144.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T23:12:09.021 INFO:teuthology.orchestra.run.smithi170.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:12:09.065 INFO:teuthology.orchestra.run.smithi144.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T23:12:09.066 INFO:teuthology.orchestra.run.smithi144.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T23:12:09.067 INFO:teuthology.orchestra.run.smithi144.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T23:12:09.147 INFO:teuthology.orchestra.run.smithi144.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T23:12:09.149 INFO:teuthology.orchestra.run.smithi144.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T23:12:09.153 INFO:teuthology.orchestra.run.smithi144.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T23:12:09.155 INFO:teuthology.orchestra.run.smithi144.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T23:12:09.157 INFO:teuthology.orchestra.run.smithi144.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T23:12:09.159 INFO:teuthology.orchestra.run.smithi144.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T23:12:09.163 INFO:teuthology.orchestra.run.smithi144.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T23:12:09.228 INFO:teuthology.orchestra.run.smithi144.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T23:12:09.230 INFO:teuthology.orchestra.run.smithi144.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T23:12:09.234 INFO:teuthology.orchestra.run.smithi144.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T23:12:09.397 INFO:teuthology.orchestra.run.smithi144.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T23:12:09.399 INFO:teuthology.orchestra.run.smithi144.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T23:12:09.401 INFO:teuthology.orchestra.run.smithi144.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T23:12:09.401 INFO:teuthology.orchestra.run.smithi144.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T23:12:09.404 INFO:teuthology.orchestra.run.smithi144.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T23:12:09.476 INFO:teuthology.orchestra.run.smithi144.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T23:12:09.500 INFO:teuthology.orchestra.run.smithi053.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-21T23:12:09.506 INFO:teuthology.orchestra.run.smithi170.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:12:09.544 INFO:teuthology.orchestra.run.smithi170.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:12:09.563 INFO:teuthology.orchestra.run.smithi170.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:12:09.564 INFO:teuthology.orchestra.run.smithi170.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:12:09.568 INFO:teuthology.orchestra.run.smithi170.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:12:09.570 INFO:teuthology.orchestra.run.smithi170.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi144.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T23:12:09.575 INFO:teuthology.orchestra.run.smithi144.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi144.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T23:12:09.576 INFO:teuthology.orchestra.run.smithi144.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:12:09.586 INFO:teuthology.orchestra.run.smithi170.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:12:09.590 INFO:teuthology.orchestra.run.smithi144.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:12:09.606 INFO:teuthology.orchestra.run.smithi144.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T23:12:09.606 INFO:teuthology.orchestra.run.smithi144.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T23:12:09.607 INFO:teuthology.orchestra.run.smithi144.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T23:12:09.608 INFO:teuthology.orchestra.run.smithi144.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T23:12:09.608 INFO:teuthology.orchestra.run.smithi144.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T23:12:09.609 INFO:teuthology.orchestra.run.smithi144.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T23:12:09.610 INFO:teuthology.orchestra.run.smithi144.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T23:12:09.627 INFO:teuthology.orchestra.run.smithi144.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:12:09.637 INFO:teuthology.orchestra.run.smithi144.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:12:09.639 INFO:teuthology.orchestra.run.smithi144.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:12:09.660 INFO:teuthology.orchestra.run.smithi144.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T23:12:09.661 INFO:teuthology.orchestra.run.smithi144.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:12:09.693 INFO:teuthology.orchestra.run.smithi144.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T23:12:09.697 INFO:teuthology.orchestra.run.smithi144.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T23:12:09.700 INFO:teuthology.orchestra.run.smithi144.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T23:12:09.702 INFO:teuthology.orchestra.run.smithi144.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T23:12:09.711 INFO:teuthology.orchestra.run.smithi144.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T23:12:09.746 INFO:teuthology.orchestra.run.smithi144.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T23:12:09.748 INFO:teuthology.orchestra.run.smithi144.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T23:12:09.750 INFO:teuthology.orchestra.run.smithi144.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T23:12:09.761 INFO:teuthology.orchestra.run.smithi144.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T23:12:09.823 INFO:teuthology.orchestra.run.smithi144.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T23:12:09.830 INFO:teuthology.orchestra.run.smithi144.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T23:12:09.831 INFO:teuthology.orchestra.run.smithi144.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T23:12:09.832 INFO:teuthology.orchestra.run.smithi144.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T23:12:09.833 INFO:teuthology.orchestra.run.smithi144.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T23:12:09.850 INFO:teuthology.orchestra.run.smithi144.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:12:09.851 INFO:teuthology.orchestra.run.smithi144.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:12:09.870 INFO:teuthology.orchestra.run.smithi053.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-21T23:12:09.904 INFO:teuthology.orchestra.run.smithi144.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T23:12:09.906 INFO:teuthology.orchestra.run.smithi144.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T23:12:09.924 INFO:teuthology.orchestra.run.smithi144.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T23:12:09.925 INFO:teuthology.orchestra.run.smithi144.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T23:12:09.926 INFO:teuthology.orchestra.run.smithi144.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T23:12:09.947 INFO:teuthology.orchestra.run.smithi053.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-21T23:12:09.986 INFO:teuthology.orchestra.run.smithi144.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T23:12:09.987 INFO:teuthology.orchestra.run.smithi144.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T23:12:10.013 INFO:teuthology.orchestra.run.smithi053.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-21T23:12:10.015 INFO:teuthology.orchestra.run.smithi053.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-21T23:12:10.075 INFO:teuthology.orchestra.run.smithi053.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-21T23:12:10.077 INFO:teuthology.orchestra.run.smithi053.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-21T23:12:10.145 INFO:teuthology.orchestra.run.smithi053.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-21T23:12:10.215 INFO:teuthology.orchestra.run.smithi170.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:12:10.218 INFO:teuthology.orchestra.run.smithi170.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:12:10.234 INFO:teuthology.orchestra.run.smithi170.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:12:11.046 INFO:teuthology.orchestra.run.smithi053.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-21T23:12:11.049 INFO:teuthology.orchestra.run.smithi053.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-21T23:12:11.120 INFO:teuthology.orchestra.run.smithi053.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-21T23:12:11.345 INFO:teuthology.orchestra.run.smithi144.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:12:11.347 INFO:teuthology.orchestra.run.smithi144.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:12:11.349 INFO:teuthology.orchestra.run.smithi144.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:12:11.634 INFO:teuthology.orchestra.run.smithi144.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:12:11.635 INFO:teuthology.orchestra.run.smithi144.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-373-g4e45e6b4-1jammy [178 kB] 2024-09-21T23:12:11.865 INFO:teuthology.orchestra.run.smithi170.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:12:11.943 INFO:teuthology.orchestra.run.smithi170.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:12:11.946 INFO:teuthology.orchestra.run.smithi170.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:12:11.947 INFO:teuthology.orchestra.run.smithi170.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:12:11.986 INFO:teuthology.orchestra.run.smithi170.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:12:12.155 INFO:teuthology.orchestra.run.smithi170.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:12:12.204 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 181 MB in 4s (45.5 MB/s) 2024-09-21T23:12:12.382 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:12:12.459 INFO:teuthology.orchestra.run.smithi053.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-21T23:12:12.483 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:12:12.488 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:12:12.572 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:12:12.639 INFO:teuthology.orchestra.run.smithi053.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-21T23:12:12.644 INFO:teuthology.orchestra.run.smithi053.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-21T23:12:12.647 INFO:teuthology.orchestra.run.smithi053.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-21T23:12:12.717 INFO:teuthology.orchestra.run.smithi053.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-21T23:12:12.845 INFO:teuthology.orchestra.run.smithi170.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:12:12.846 INFO:teuthology.orchestra.run.smithi170.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:12:12.863 INFO:teuthology.orchestra.run.smithi170.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:12:12.903 INFO:teuthology.orchestra.run.smithi170.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:12:12.913 INFO:teuthology.orchestra.run.smithi170.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:12:12.915 INFO:teuthology.orchestra.run.smithi170.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:12:12.957 INFO:teuthology.orchestra.run.smithi053.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-21T23:12:12.975 INFO:teuthology.orchestra.run.smithi170.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:12:13.214 INFO:teuthology.orchestra.run.smithi170.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:12:13.215 INFO:teuthology.orchestra.run.smithi170.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:12:13.359 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:12:13.385 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:12:13.442 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:12:13.762 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:12:13.788 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:12:13.829 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:12:14.015 INFO:teuthology.orchestra.run.smithi053.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-21T23:12:14.016 INFO:teuthology.orchestra.run.smithi053.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-21T23:12:14.033 INFO:teuthology.orchestra.run.smithi053.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-21T23:12:14.096 INFO:teuthology.orchestra.run.smithi053.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-21T23:12:14.142 INFO:teuthology.orchestra.run.smithi053.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-21T23:12:14.145 INFO:teuthology.orchestra.run.smithi053.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-21T23:12:14.236 INFO:teuthology.orchestra.run.smithi053.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-21T23:12:14.249 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:12:14.275 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:14.324 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:14.599 INFO:teuthology.orchestra.run.smithi053.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-21T23:12:14.600 INFO:teuthology.orchestra.run.smithi053.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-21T23:12:14.752 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:12:14.778 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:14.819 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:14.823 INFO:teuthology.orchestra.run.smithi170.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:12:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:12:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:12:15.139 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:12:15.160 INFO:teuthology.orchestra.run.smithi170.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:12:15.162 INFO:teuthology.orchestra.run.smithi170.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-373-g4e45e6b4-1jammy [178 kB] 2024-09-21T23:12:15.165 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:15.197 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:15.542 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:12:15.568 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:12:15.608 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:12:15.740 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 181 MB in 7s (25.5 MB/s) 2024-09-21T23:12:15.902 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:12:15.928 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librados2. 2024-09-21T23:12:15.954 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:15.994 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:16.002 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:12:16.008 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:12:16.092 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:12:16.431 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:12:16.445 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:12:16.457 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:16.472 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:12:16.497 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:16.511 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:12:16.900 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:12:16.926 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:16.967 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:17.031 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:12:17.058 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:12:17.098 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:12:17.252 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:12:17.268 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:17.311 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:17.601 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:12:17.627 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:17.638 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:12:17.665 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:17.677 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:17.705 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:17.927 INFO:teuthology.orchestra.run.smithi053.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-21T23:12:17.929 INFO:teuthology.orchestra.run.smithi053.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-21T23:12:17.931 INFO:teuthology.orchestra.run.smithi053.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-21T23:12:17.949 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:12:17.965 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:17.999 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:18.071 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:12:18.097 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:18.138 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:18.286 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:12:18.311 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:18.352 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:18.457 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:12:18.484 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:18.516 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:18.663 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:12:18.690 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:12:18.730 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:12:18.861 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:12:18.887 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:12:18.927 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:12:19.058 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:12:19.084 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:12:19.125 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:12:19.263 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librados2. 2024-09-21T23:12:19.290 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:19.330 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:19.362 INFO:teuthology.orchestra.run.smithi053.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-21T23:12:19.364 INFO:teuthology.orchestra.run.smithi053.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.1.1-373-g4e45e6b4-1jammy [178 kB] 2024-09-21T23:12:19.385 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:12:19.401 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:19.435 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:19.747 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:12:19.773 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:12:19.783 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:12:19.809 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:19.813 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:12:19.849 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:19.951 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 181 MB in 11s (16.2 MB/s) 2024-09-21T23:12:20.108 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:12:20.112 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T23:12:20.123 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:12:20.166 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:12:20.215 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T23:12:20.220 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T23:12:20.302 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:12:20.302 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:12:20.328 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:20.369 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:20.469 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:12:20.495 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:12:20.536 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:12:20.655 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:12:20.670 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:20.672 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T23:12:20.699 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T23:12:20.713 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:20.738 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:12:20.855 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:12:20.871 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:12:20.905 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:12:21.041 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:12:21.067 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:21.107 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:21.142 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T23:12:21.169 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T23:12:21.209 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:12:21.210 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:12:21.235 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:12:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:12:21.351 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:12:21.366 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:21.401 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:21.520 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package zip. 2024-09-21T23:12:21.546 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:12:21.586 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:12:21.654 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:12:21.655 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T23:12:21.669 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:21.682 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:21.721 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:21.730 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:21.873 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package unzip. 2024-09-21T23:12:21.899 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:12:21.939 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:12:22.084 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T23:12:22.110 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:22.142 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:22.216 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:12:22.242 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:12:22.282 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:12:22.335 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:12:22.361 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:12:22.401 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:12:22.610 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:12:22.637 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:12:22.676 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:12:22.712 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T23:12:22.739 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T23:12:22.778 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:12:22.801 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:12:22.828 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:22.868 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:22.937 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:12:22.964 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:23.013 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:23.131 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T23:12:23.158 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T23:12:23.198 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:12:23.357 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:12:23.384 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:12:23.424 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:12:23.501 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librados2. 2024-09-21T23:12:23.528 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:23.567 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:23.693 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:12:23.708 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:12:23.743 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:12:24.004 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:12:24.019 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:12:24.029 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librbd1. 2024-09-21T23:12:24.054 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:12:24.056 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:24.087 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:24.112 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:12:24.139 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:24.195 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:24.391 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:12:24.417 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:12:24.457 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:12:24.515 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:12:24.531 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T23:12:24.542 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:12:24.558 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:24.590 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:12:24.598 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:24.777 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:12:24.803 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:12:24.843 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:12:24.875 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rados. 2024-09-21T23:12:24.885 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:12:24.902 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:24.912 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:24.942 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:24.943 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:25.054 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package zip. 2024-09-21T23:12:25.070 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:12:25.121 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:12:25.261 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T23:12:25.289 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:25.305 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:12:25.328 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:25.332 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:25.371 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:25.424 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package unzip. 2024-09-21T23:12:25.451 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:12:25.491 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:12:25.555 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T23:12:25.580 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:25.630 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:25.894 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:12:25.921 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:12:25.950 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T23:12:25.961 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:12:25.976 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:26.033 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:26.344 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T23:12:26.360 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:12:26.371 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T23:12:26.387 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:26.411 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:12:26.427 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:26.714 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T23:12:26.741 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T23:12:26.768 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:12:26.781 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T23:12:26.795 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:26.844 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:26.963 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:12:26.978 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:27.013 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:27.049 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T23:12:27.065 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:27.108 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:27.349 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:12:27.375 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:12:27.453 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T23:12:27.464 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:12:27.479 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T23:12:27.490 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:12:27.531 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:12:27.534 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:12:27.574 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:12:27.809 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:12:27.835 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:12:27.875 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:12:27.893 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:12:27.913 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T23:12:27.920 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:27.929 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T23:12:27.960 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:27.963 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:12:28.195 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:12:28.221 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:12:28.233 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T23:12:28.248 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T23:12:28.262 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:12:28.291 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:12:28.413 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:12:28.440 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:28.488 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:28.564 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:12:28.591 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:12:28.619 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua5.1. 2024-09-21T23:12:28.631 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:12:28.646 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T23:12:28.694 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:12:28.909 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:12:28.935 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:12:28.976 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:12:29.014 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua-any. 2024-09-21T23:12:29.041 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-21T23:12:29.081 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T23:12:29.245 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:12:29.271 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:12:29.291 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package zip. 2024-09-21T23:12:29.307 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-21T23:12:29.312 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:12:29.350 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T23:12:29.615 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:12:29.641 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:12:29.669 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package unzip. 2024-09-21T23:12:29.681 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:12:29.696 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T23:12:29.736 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:12:29.943 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:12:29.970 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:29.984 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:12:30.011 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:12:30.019 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:30.051 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:12:30.132 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package luarocks. 2024-09-21T23:12:30.159 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T23:12:30.198 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:12:30.312 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:12:30.339 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:12:30.370 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:12:30.566 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:12:30.581 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librgw2. 2024-09-21T23:12:30.581 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:12:30.608 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:30.616 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:12:30.647 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:30.724 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:12:30.751 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:12:30.791 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:12:30.894 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:12:30.921 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:12:30.960 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:12:31.077 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:12:31.104 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:12:31.135 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:12:31.225 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T23:12:31.241 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:31.283 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:31.288 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:12:31.315 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:12:31.355 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:12:31.430 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:12:31.456 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:12:31.496 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:12:31.586 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T23:12:31.613 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T23:12:31.644 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:12:31.658 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:12:31.685 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:12:31.725 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:12:31.816 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:12:31.843 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:12:31.883 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:12:31.939 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T23:12:31.966 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:31.986 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:12:32.005 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:32.013 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:12:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:12:32.211 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:12:32.237 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:12:32.278 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:12:32.372 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:12:32.399 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:12:32.434 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-common. 2024-09-21T23:12:32.439 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:12:32.460 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:32.500 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:32.547 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:12:32.573 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:12:32.605 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:12:32.742 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:12:32.769 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:12:32.808 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:12:32.883 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:12:32.910 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:12:32.950 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:12:33.086 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:12:33.113 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:12:33.144 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:12:33.295 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:12:33.321 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:12:33.364 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:12:33.414 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:12:33.440 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:12:33.480 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:12:33.656 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:12:33.682 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:12:33.722 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:12:33.784 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:12:33.811 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:12:33.851 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:12:33.905 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-base. 2024-09-21T23:12:33.932 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:33.983 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:34.025 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:12:34.052 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:12:34.092 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:12:34.162 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:12:34.189 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:12:34.229 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:12:34.412 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:12:34.427 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:12:34.470 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:12:34.532 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:12:34.558 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:12:34.568 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T23:12:34.595 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T23:12:34.607 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:12:34.634 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:12:34.764 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:12:34.791 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:12:34.831 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:12:34.885 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:12:34.911 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:12:34.929 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T23:12:34.945 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T23:12:34.953 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:12:34.979 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:12:35.101 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:12:35.127 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:12:35.193 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:12:35.255 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:12:35.273 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T23:12:35.281 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:12:35.300 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T23:12:35.330 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:12:35.331 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:12:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:12:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:12:35.554 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:12:35.616 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:12:35.634 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T23:12:35.642 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:12:35.661 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T23:12:35.682 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:12:35.701 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:12:35.917 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:12:35.944 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:12:35.983 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:12:35.985 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:12:36.012 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:12:36.021 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T23:12:36.048 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T23:12:36.052 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:12:36.087 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:12:36.312 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:12:36.338 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:12:36.374 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T23:12:36.378 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:12:36.400 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T23:12:36.405 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:12:36.431 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:12:36.440 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T23:12:36.475 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:12:36.673 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:12:36.688 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:12:36.710 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-portend. 2024-09-21T23:12:36.723 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:12:36.737 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-21T23:12:36.775 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:12:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T23:12:36.801 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:12:36.833 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:12:37.018 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:12:37.033 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:12:37.067 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:12:37.087 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T23:12:37.114 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T23:12:37.119 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:12:37.146 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:12:37.154 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T23:12:37.186 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:12:37.337 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:12:37.364 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:12:37.404 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:12:37.440 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T23:12:37.467 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T23:12:37.497 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:12:37.507 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:12:37.524 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:12:37.564 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:12:37.665 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:12:37.680 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:37.715 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:38.067 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:12:38.087 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:12:38.093 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:12:38.094 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T23:12:38.113 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:38.121 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T23:12:38.133 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:12:38.145 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:38.160 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T23:12:38.398 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:12:38.425 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:12:38.462 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:12:38.464 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:12:38.471 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T23:12:38.488 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:12:38.499 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T23:12:38.528 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:12:38.538 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T23:12:38.801 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:12:38.824 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T23:12:38.827 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:12:38.840 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:12:38.841 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T23:12:38.866 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:12:38.867 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:12:38.882 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:12:38.906 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:12:39.212 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:12:39.238 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:39.278 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:12:39.305 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:12:39.344 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:12:39.570 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:39.622 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:12:39.649 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:12:39.689 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:12:39.927 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-mako. 2024-09-21T23:12:39.940 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:12:39.954 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T23:12:39.966 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:12:40.006 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:40.009 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:12:40.035 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:12:40.075 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:12:40.289 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T23:12:40.316 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T23:12:40.347 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:12:40.395 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:12:40.422 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:12:40.461 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:12:40.600 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T23:12:40.604 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:12:40.627 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T23:12:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:40.666 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:12:40.670 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:40.748 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:12:40.763 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:12:40.806 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:12:40.952 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-webob. 2024-09-21T23:12:40.979 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T23:12:41.018 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:12:41.364 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T23:12:41.390 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T23:12:41.433 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:12:41.716 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T23:12:41.732 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph. 2024-09-21T23:12:41.743 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T23:12:41.758 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:41.783 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:12:41.798 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:41.985 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:12:42.012 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:42.051 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:42.119 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-paste. 2024-09-21T23:12:42.146 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T23:12:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:12:42.177 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:42.185 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:12:42.217 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:42.432 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:12:42.458 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:42.497 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T23:12:42.498 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:42.524 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T23:12:42.554 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:12:42.563 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:12:42.569 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:42.604 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:42.785 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:12:42.811 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:12:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:12:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T23:12:42.893 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T23:12:42.933 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:12:42.982 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:12:43.009 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:43.048 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:43.112 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:12:43.128 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:12:43.171 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:12:43.244 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T23:12:43.271 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T23:12:43.311 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T23:12:43.359 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:12:43.375 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:12:43.418 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:12:43.507 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:12:43.533 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:43.573 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:43.605 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T23:12:43.621 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T23:12:43.655 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:12:43.762 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:12:43.778 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:12:43.820 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:12:43.969 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:12:43.995 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:44.002 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T23:12:44.018 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T23:12:44.035 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:44.059 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T23:12:44.363 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-idna. 2024-09-21T23:12:44.390 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T23:12:44.429 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:12:44.631 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:12:44.657 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:44.688 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:44.732 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T23:12:44.759 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T23:12:44.798 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:12:44.851 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:12:44.878 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:12:44.917 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:12:45.119 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-requests. 2024-09-21T23:12:45.146 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T23:12:45.185 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:12:45.229 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:12:45.255 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:12:45.295 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:12:45.530 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T23:12:45.557 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T23:12:45.596 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:12:45.615 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:12:45.641 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:45.698 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:45.775 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph. 2024-09-21T23:12:45.802 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:45.842 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:45.882 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T23:12:45.909 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:45.949 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:46.035 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:12:46.061 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:12:46.101 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:12:46.178 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:12:46.204 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:46.245 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:46.320 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T23:12:46.346 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:12:46.347 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:46.361 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:12:46.387 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:46.404 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:12:46.572 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:12:46.588 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:46.622 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:46.690 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:12:46.706 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:46.707 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T23:12:46.733 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T23:12:46.740 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:46.773 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:12:47.009 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:12:47.036 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:47.075 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:47.076 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T23:12:47.103 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T23:12:47.142 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T23:12:47.403 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:12:47.430 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:12:47.453 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T23:12:47.470 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:12:47.481 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:47.512 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:47.814 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:12:47.841 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:12:47.880 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:12:47.898 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T23:12:47.925 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:47.965 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:48.021 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:12:48.050 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:12:48.087 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:12:48.599 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:12:48.626 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:12:48.665 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:12:48.860 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T23:12:48.887 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:48.926 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:49.036 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:12:49.053 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:12:49.063 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:12:49.080 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:12:49.119 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:12:49.119 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:12:49.414 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:12:49.440 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:12:49.447 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:12:49.473 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:12:49.480 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:12:49.513 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:12:49.808 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:12:49.835 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:49.875 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:50.138 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph. 2024-09-21T23:12:50.166 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:50.195 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:12:50.204 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:50.222 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:12:50.255 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:12:50.261 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:12:50.283 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:50.407 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:50.491 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T23:12:50.518 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:50.557 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:50.564 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:12:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:12:50.630 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:12:50.919 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T23:12:50.946 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:50.985 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:51.001 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:12:51.028 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:51.067 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:51.361 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:12:51.379 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package cephadm. 2024-09-21T23:12:51.388 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:12:51.407 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:51.427 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:12:51.446 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:51.724 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T23:12:51.730 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:12:51.740 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T23:12:51.758 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:12:51.773 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T23:12:51.797 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:12:52.076 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T23:12:52.092 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:12:52.126 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:12:52.142 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:12:52.169 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:12:52.208 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:12:52.364 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:12:52.379 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:12:52.422 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:12:52.512 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:12:52.539 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:12:52.578 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:12:52.915 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:12:52.935 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:12:52.942 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:12:52.961 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:12:52.982 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:12:53.001 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:12:53.206 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-babel. 2024-09-21T23:12:53.234 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T23:12:53.272 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:12:53.285 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:12:53.312 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:12:53.352 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:12:53.363 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:12:53.390 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:12:53.421 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:12:53.601 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T23:12:53.613 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:12:53.628 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T23:12:53.640 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:12:53.667 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:12:53.679 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:12:53.707 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:12:53.734 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:12:53.798 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:12:53.991 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:12:54.012 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T23:12:54.018 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:12:54.039 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:54.057 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:12:54.078 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:54.374 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T23:12:54.401 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T23:12:54.432 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T23:12:54.457 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:12:54.485 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:54.627 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:54.693 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-routes. 2024-09-21T23:12:54.705 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:12:54.720 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T23:12:54.733 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:54.759 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:12:54.771 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:55.054 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T23:12:55.070 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:55.112 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:55.349 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:12:55.377 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:12:55.416 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:12:55.605 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:12:55.632 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:12:55.671 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:12:55.728 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:12:55.755 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:12:55.786 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:12:55.983 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:12:56.010 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:12:56.038 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package jq. 2024-09-21T23:12:56.049 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:12:56.054 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:12:56.097 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:12:56.325 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T23:12:56.342 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T23:12:56.349 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package socat. 2024-09-21T23:12:56.365 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:12:56.369 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:12:56.383 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:12:56.397 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:12:56.433 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:12:56.436 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:12:56.744 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:12:56.747 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:12:56.760 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:12:56.774 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:12:56.803 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:12:56.832 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:12:56.904 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T23:12:56.931 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T23:12:56.970 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:12:57.101 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:12:57.117 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:12:57.157 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:12:57.159 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:12:57.184 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:12:57.223 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:57.349 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T23:12:57.377 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T23:12:57.415 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:12:57.471 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:12:57.498 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:12:57.537 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:12:57.727 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T23:12:57.754 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T23:12:57.793 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:12:57.849 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:12:57.877 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:12:57.915 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:12:58.236 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:12:58.263 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:12:58.302 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:12:58.527 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T23:12:58.555 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:58.678 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:58.982 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:12:58.999 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:59.041 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:59.369 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:12:59.396 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:12:59.519 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:12:59.745 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:12:59.772 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:12:59.789 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:12:59.811 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:12:59.816 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:12:59.855 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:12:59.867 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T23:12:59.895 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T23:12:59.934 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T23:13:00.106 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:13:00.121 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:00.158 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package jq. 2024-09-21T23:13:00.164 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:00.185 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:13:00.224 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:13:00.262 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T23:13:00.289 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T23:13:00.328 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:13:00.477 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package socat. 2024-09-21T23:13:00.492 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:13:00.505 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:13:00.520 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:13:00.544 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:13:00.559 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:13:00.665 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T23:13:00.693 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T23:13:00.731 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:13:00.847 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:13:00.874 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:13:00.906 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:13:00.913 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:13:00.933 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:13:00.972 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:13:01.051 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T23:13:01.079 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-21T23:13:01.118 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T23:13:01.285 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:13:01.301 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:01.317 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:13:01.343 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:01.345 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:13:01.384 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:13:01.438 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T23:13:01.466 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T23:13:01.504 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T23:13:01.806 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:13:01.808 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T23:13:01.833 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:13:01.835 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T23:13:01.872 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:13:01.874 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:13:02.133 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:13:02.149 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:13:02.185 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T23:13:02.192 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:13:02.213 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T23:13:02.252 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T23:13:02.563 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T23:13:02.591 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T23:13:02.620 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:13:02.630 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:13:02.648 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:13:02.687 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:13:02.989 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:13:03.017 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:13:03.056 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:13:03.335 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T23:13:03.351 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:13:03.362 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:13:03.378 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:13:03.402 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:03.410 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:13:03.681 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:13:03.708 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:13:03.737 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:13:03.738 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T23:13:03.747 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:03.765 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:13:03.766 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T23:13:03.804 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:13:03.804 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:13:04.108 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T23:13:04.132 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:13:04.135 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:13:04.159 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:13:04.166 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:13:04.198 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:13:04.285 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:13:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:04.351 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:04.402 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package jq. 2024-09-21T23:13:04.419 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T23:13:04.460 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T23:13:04.543 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:13:04.571 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:13:04.610 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:13:04.646 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:13:04.661 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:13:04.695 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:13:04.763 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package socat. 2024-09-21T23:13:04.791 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T23:13:04.837 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:13:05.034 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:13:05.061 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:13:05.097 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:13:05.100 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:13:05.125 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:13:05.157 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T23:13:05.164 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:13:05.174 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T23:13:05.215 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:13:05.425 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:13:05.441 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:13:05.471 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:13:05.475 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:13:05.498 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:13:05.537 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:13:05.620 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-test. 2024-09-21T23:13:05.648 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:05.694 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:05.745 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:13:05.772 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:13:05.811 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:13:05.960 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:13:05.987 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:13:06.026 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:13:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:13:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:13:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:13:06.312 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:13:06.328 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:13:06.370 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:13:06.509 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:13:06.524 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:13:06.567 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:13:06.799 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:13:06.814 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:13:06.848 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:13:07.137 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:13:07.162 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:13:07.201 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:13:07.338 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:13:07.366 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:07.404 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:07.521 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:13:07.549 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:13:07.588 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:13:07.899 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:13:07.927 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:13:07.966 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:13:08.049 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T23:13:08.077 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-21T23:13:08.116 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:08.191 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:13:08.207 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:08.240 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:08.319 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:13:08.346 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:13:08.394 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:13:08.402 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T23:13:08.419 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:08.452 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:08.493 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-nbd. 2024-09-21T23:13:08.521 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:08.559 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:08.697 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:13:08.724 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:13:08.747 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T23:13:08.763 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:13:08.775 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T23:13:08.830 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:13:08.856 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:13:09.106 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:13:09.143 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T23:13:09.172 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T23:13:09.209 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:13:09.233 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:13:09.270 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:13:09.297 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:13:09.336 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:13:09.515 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:13:09.547 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T23:13:09.575 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T23:13:09.613 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:13:09.672 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:13:09.700 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:13:09.739 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:13:09.847 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:13:10.002 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package pkg-config. 2024-09-21T23:13:10.030 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T23:13:10.042 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:13:10.068 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:13:10.069 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:13:10.108 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:13:10.266 INFO:teuthology.orchestra.run.smithi144.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:13:10.321 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T23:13:10.338 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T23:13:10.342 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:10.379 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:13:10.454 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:13:10.482 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:13:10.520 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:13:10.789 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:13:10.805 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:13:10.839 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:13:10.841 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T23:13:10.869 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T23:13:10.907 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T23:13:10.909 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user cephadm....done 2024-09-21T23:13:11.065 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:13:11.193 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T23:13:11.210 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T23:13:11.251 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T23:13:11.317 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:13:11.527 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:13:11.538 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T23:13:11.554 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T23:13:11.555 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:11.560 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:13:11.585 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:11.588 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:13:11.711 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:13:11.866 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T23:13:11.894 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T23:13:11.932 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T23:13:11.962 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:13:12.222 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:13:12.252 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-py. 2024-09-21T23:13:12.280 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-21T23:13:12.318 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T23:13:12.348 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:13:12.405 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:13:12.421 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:12.463 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:12.631 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T23:13:12.649 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:13:12.659 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T23:13:12.697 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:13:12.732 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-nbd. 2024-09-21T23:13:12.749 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:12.790 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:13.001 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:13:13.103 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:13:13.159 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T23:13:13.187 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T23:13:13.225 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:13:13.278 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:13:13.371 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:13:13.497 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:13:13.520 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-toml. 2024-09-21T23:13:13.548 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-21T23:13:13.554 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:13:13.586 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T23:13:13.754 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:13:13.806 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:13:13.839 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T23:13:13.856 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T23:13:13.889 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:13:14.054 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:13:14.073 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:13:14.235 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T23:13:14.263 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T23:13:14.301 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:13:14.383 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:13:14.488 INFO:teuthology.orchestra.run.smithi170.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:13:14.557 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:14.603 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T23:13:14.620 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T23:13:14.634 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:14.661 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:13:14.927 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:13:15.061 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:13:15.098 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user cephadm....done 2024-09-21T23:13:15.187 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:13:15.268 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:13:15.313 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:13:15.416 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package radosgw. 2024-09-21T23:13:15.444 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:15.473 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:15.539 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:13:15.581 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:13:15.783 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:13:15.909 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:13:16.168 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:13:16.258 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:13:16.287 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:13:16.293 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T23:13:16.310 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:16.352 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:16.359 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:13:16.411 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:13:16.511 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:13:16.537 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:13:16.612 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package rbd-nbd. 2024-09-21T23:13:16.629 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../118-rbd-nbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-21T23:13:16.670 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:16.847 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:13:16.878 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:13:16.976 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T23:13:17.005 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:13:17.207 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:13:17.267 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T23:13:17.289 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:13:17.377 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T23:13:17.467 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:13:17.557 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:13:17.637 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T23:13:17.743 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:13:17.850 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:13:17.976 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:13:17.981 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T23:13:18.012 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:13:18.263 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:13:18.416 INFO:teuthology.orchestra.run.smithi053.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T23:13:18.478 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:18.547 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:13:18.653 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:13:18.807 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:18.937 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:13:18.970 INFO:teuthology.orchestra.run.smithi053.stdout:Adding system user cephadm....done 2024-09-21T23:13:19.063 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:13:19.126 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T23:13:19.166 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:13:19.189 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:13:19.292 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:13:19.386 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T23:13:19.418 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:13:19.490 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:13:19.544 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:13:19.628 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T23:13:19.755 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T23:13:19.812 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:13:19.875 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:13:20.005 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T23:13:20.109 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:13:20.160 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:13:20.193 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:13:20.249 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:13:20.265 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T23:13:20.366 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T23:13:20.375 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:13:20.486 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:13:20.676 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T23:13:20.743 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:13:20.779 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:13:20.852 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:13:21.037 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T23:13:21.112 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:13:21.147 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:13:21.296 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T23:13:21.371 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:13:21.415 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:13:21.583 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:21.606 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T23:13:21.698 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:13:21.799 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:13:21.857 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T23:13:21.892 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:13:22.108 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T23:13:22.219 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:13:22.368 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T23:13:22.475 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:13:22.635 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:22.718 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:13:22.853 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:13:22.945 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T23:13:22.979 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:13:23.071 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T23:13:23.205 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T23:13:23.271 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:13:23.331 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T23:13:23.424 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:13:23.574 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T23:13:23.673 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:13:23.692 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:23.827 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:13:23.899 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:13:23.935 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T23:13:23.963 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T23:13:24.019 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T23:13:24.095 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:13:24.120 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T23:13:24.221 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:13:24.267 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:13:24.347 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:13:24.479 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T23:13:24.552 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:13:24.605 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T23:13:24.640 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:13:24.873 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T23:13:24.891 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:13:24.912 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:13:25.126 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:25.171 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:13:25.174 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T23:13:25.254 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:13:25.314 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:25.501 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T23:13:25.555 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:13:25.599 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:13:25.626 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T23:13:25.806 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:13:25.925 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:13:26.057 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:13:26.308 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:13:26.320 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T23:13:26.559 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:13:26.579 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T23:13:26.705 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T23:13:26.827 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:13:26.831 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T23:13:26.954 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:13:27.031 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:13:27.116 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T23:13:27.238 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:13:27.266 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:13:27.400 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:13:27.517 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T23:13:27.656 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:13:27.668 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:13:27.768 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T23:13:27.819 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:13:27.945 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:13:27.957 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:13:28.136 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T23:13:28.225 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:13:28.247 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:13:28.351 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:13:28.429 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T23:13:28.477 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:13:28.498 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:13:28.567 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:13:28.732 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:28.780 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T23:13:28.829 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:13:28.858 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:13:28.955 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:13:29.040 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T23:13:29.160 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:13:29.165 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:29.331 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:13:29.411 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:13:29.467 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T23:13:29.574 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:13:29.671 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:13:29.700 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:13:29.793 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T23:13:29.922 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:13:29.959 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:30.085 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:13:30.173 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:13:30.420 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:13:30.432 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:13:30.558 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:13:30.721 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:13:30.851 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:13:30.969 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:13:30.997 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:13:31.057 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T23:13:31.123 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:13:31.254 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:13:31.292 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:31.383 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:13:31.426 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T23:13:31.555 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:13:31.633 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:13:31.686 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T23:13:31.803 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T23:13:31.814 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:13:31.915 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:13:31.929 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T23:13:31.978 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:13:32.041 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:13:32.131 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:13:32.231 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T23:13:32.338 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:13:32.368 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:13:32.503 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:13:32.515 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T23:13:32.640 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:13:32.766 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:32.893 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T23:13:32.895 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:13:32.967 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:13:33.093 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:33.147 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:13:33.185 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T23:13:33.219 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:33.248 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:13:33.354 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:13:33.445 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T23:13:33.499 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:33.625 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:13:33.682 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:33.704 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T23:13:33.791 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:33.909 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:13:33.951 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:13:33.988 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T23:13:34.194 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:34.236 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:13:34.239 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T23:13:34.320 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:34.438 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:34.499 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T23:13:34.504 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:13:34.630 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:13:34.633 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T23:13:34.643 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:13:34.643 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:13:34.898 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:13:34.918 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T23:13:35.044 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T23:13:35.139 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:35.337 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T23:13:35.357 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:35.458 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:13:35.467 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:13:35.663 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T23:13:35.836 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:13:35.939 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T23:13:36.064 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T23:13:36.191 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T23:13:36.221 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:13:36.279 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T23:13:36.489 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:13:36.517 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T23:13:36.643 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T23:13:36.662 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:36.763 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:36.808 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:13:36.864 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:36.934 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:36.990 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:37.045 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T23:13:37.069 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:37.116 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:37.195 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:13:37.242 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:37.313 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T23:13:37.439 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T23:13:37.490 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:37.607 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:37.689 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T23:13:37.734 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:13:37.790 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T23:13:37.793 INFO:teuthology.orchestra.run.smithi144.stdout:Adding group ceph....done 2024-09-21T23:13:38.018 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:38.100 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T23:13:38.192 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:38.192 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user ceph....done 2024-09-21T23:13:38.271 INFO:teuthology.orchestra.run.smithi144.stdout:Setting system user ceph properties....done 2024-09-21T23:13:38.279 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:38.283 INFO:teuthology.orchestra.run.smithi144.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:13:38.410 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T23:13:38.417 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:13:38.474 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:13:38.474 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:13:38.661 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T23:13:38.718 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:13:38.762 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T23:13:38.980 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:39.037 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T23:13:39.081 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:39.191 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:39.218 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:39.314 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T23:13:39.316 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:13:39.344 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:39.659 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T23:13:39.837 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:13:39.837 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:13:40.052 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T23:13:40.304 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:40.320 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T23:13:40.546 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:40.622 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:13:40.647 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:40.673 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T23:13:40.764 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:40.782 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:40.874 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:40.908 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.000 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.043 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T23:13:41.089 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.126 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.298 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:13:41.298 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:13:41.379 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.505 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.606 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T23:13:41.652 INFO:teuthology.orchestra.run.smithi170.stdout:Adding group ceph....done 2024-09-21T23:13:41.758 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.865 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:41.972 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:13:41.973 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:13:42.008 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:42.029 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user ceph....done 2024-09-21T23:13:42.135 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:42.172 INFO:teuthology.orchestra.run.smithi170.stdout:Setting system user ceph properties....done 2024-09-21T23:13:42.186 INFO:teuthology.orchestra.run.smithi170.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:13:42.320 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:13:42.344 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:13:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T23:13:42.467 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:42.650 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:13:42.750 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:13:42.750 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:13:42.861 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:42.987 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:43.144 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:43.227 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:43.270 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:43.362 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:43.493 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:43.513 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:43.607 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T23:13:43.715 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:13:43.715 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:13:43.730 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:13:43.730 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:13:44.189 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:44.197 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:44.340 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:44.441 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:44.549 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:13:44.567 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:44.919 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:44.928 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:13:45.019 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:45.051 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:13:45.112 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:45.129 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up rbd-nbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:45.224 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:13:45.255 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:45.381 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:45.472 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:13:45.472 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:13:45.507 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:45.911 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:46.018 INFO:teuthology.orchestra.run.smithi053.stdout:Adding group ceph....done 2024-09-21T23:13:46.274 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:13:46.275 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:13:46.282 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:13:46.412 INFO:teuthology.orchestra.run.smithi053.stdout:Adding system user ceph....done 2024-09-21T23:13:46.581 INFO:teuthology.orchestra.run.smithi053.stdout:Setting system user ceph properties....done 2024-09-21T23:13:46.593 INFO:teuthology.orchestra.run.smithi053.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T23:13:46.738 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T23:13:46.764 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:47.048 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:13:47.049 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:13:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T23:13:47.347 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:13:47.351 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:47.525 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:47.527 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:47.628 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:47.651 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:47.811 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:48.028 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:13:48.028 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:13:48.077 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:13:48.077 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T23:13:48.512 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:48.547 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:48.663 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:48.864 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:48.873 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T23:13:49.098 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:49.340 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:49.451 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:13:49.560 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:13:49.584 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:13:49.584 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T23:13:49.721 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:13:50.092 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:50.336 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:13:50.336 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T23:13:50.505 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:13:50.801 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:51.072 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:13:51.072 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T23:13:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:13:51.154 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:51.588 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:51.713 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:51.865 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:52.069 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:13:52.069 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T23:13:52.632 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:52.893 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:53.044 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:53.195 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-21T23:13:53.540 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T23:13:53.684 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T23:13:53.938 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T23:13:55.127 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T23:13:55.757 INFO:teuthology.orchestra.run.smithi053.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T23:13:55.761 DEBUG:teuthology.parallel:result is None 2024-09-21T23:13:55.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:13:55.910 DEBUG:teuthology.orchestra.run.smithi053:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:13:55.929 INFO:teuthology.orchestra.run.smithi053.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:13:55.929 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:13:55.929 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:13:55.931 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:13:56.068 DEBUG:teuthology.orchestra.run.smithi144:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:13:56.087 INFO:teuthology.orchestra.run.smithi144.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:13:56.088 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:13:56.088 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:13:56.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T23:13:56.226 DEBUG:teuthology.orchestra.run.smithi170:> dpkg-query -W -f '${Version}' ceph 2024-09-21T23:13:56.245 INFO:teuthology.orchestra.run.smithi170.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:13:56.245 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-21T23:13:56.245 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-21T23:13:56.247 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T23:13:56.247 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:56.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:13:56.262 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:13:56.262 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:13:56.276 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:13:56.276 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T23:13:56.300 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T23:13:56.300 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:56.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:13:56.317 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:13:56.374 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:13:56.374 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:13:56.387 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:13:56.442 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:13:56.442 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T23:13:56.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T23:13:56.510 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T23:13:56.511 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:56.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:13:56.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:13:56.581 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:13:56.581 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:13:56.594 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:13:56.650 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:13:56.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T23:13:56.664 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T23:13:56.718 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T23:13:56.719 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:56.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:13:56.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:13:56.786 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:13:56.786 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:13:56.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:13:56.855 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:13:56.855 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T23:13:56.868 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T23:13:56.921 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T23:13:56.991 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T23:13:56.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2024-09-21T23:13:56.993 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2024-09-21T23:13:56.995 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2024-09-21T23:13:57.008 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T23:13:57.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:13:57.051 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:13:57.053 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T23:13:57.064 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T23:13:57.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:13:57.107 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:13:57.112 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T23:13:57.132 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T23:13:57.133 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T23:13:57.133 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914511', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 1478, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '24/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOVINPiRSwLqQnBj7mKrhyVes1+mXwAb/TjYPRWSp8QguwEf/1DEpP248nae+R374DfPV3QlMBtHi3xG6GU3fq0=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJ72l0rICXir5VJ/uMGGuBOISpYaveP3zwcYT1/tcD+yfFixB/bh5TsxJ/nk6r0dZaqSkwFOI1PY18VoXFZh7Y=', 'smithi170.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwXtcli5i3JH3EE0aBlbr+l1kgAJnFV+kbJ2ofzRgObTFnLEZXMrywjL5X79qd/vPuvoIHGKF6QYcu/Hafhi80='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T23:13:57.133 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:13:57.163 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:13:57.168 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T23:13:57.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:13:57.211 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:13:57.216 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T23:13:57.228 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:57.228 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T23:13:57.274 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T23:13:57.274 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 23:08:20.741917176 +0000 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 23:08:20.493919391 +0000 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 23:08:20.493919391 +0000 2024-09-21T23:13:57.326 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-09-21T23:13:57.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T23:13:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T23:13:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T23:13:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000326565 s, 1.6 MB/s 2024-09-21T23:13:57.382 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T23:13:57.431 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 902 Links: 1 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 23:08:21.261912532 +0000 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 23:08:21.009914783 +0000 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 23:08:21.009914783 +0000 2024-09-21T23:13:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-09-21T23:13:57.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T23:13:57.536 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T23:13:57.536 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T23:13:57.537 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000316096 s, 1.6 MB/s 2024-09-21T23:13:57.538 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T23:13:57.587 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 23:08:21.745908208 +0000 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 23:08:21.493910459 +0000 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 23:08:21.493910459 +0000 2024-09-21T23:13:57.634 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-09-21T23:13:57.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T23:13:57.688 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T23:13:57.689 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T23:13:57.689 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000279693 s, 1.8 MB/s 2024-09-21T23:13:57.690 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T23:13:57.739 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-09-21T23:13:57.786 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T23:13:57.786 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:57.786 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-09-21T23:13:57.786 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:57.786 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 23:08:21.985906065 +0000 2024-09-21T23:13:57.786 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 23:08:21.981906100 +0000 2024-09-21T23:13:57.786 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 23:08:21.981906100 +0000 2024-09-21T23:13:57.787 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-09-21T23:13:57.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T23:13:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T23:13:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T23:13:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000315667 s, 1.6 MB/s 2024-09-21T23:13:57.845 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T23:13:57.899 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:13:57.899 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:13:57.899 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T23:13:57.905 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T23:13:57.905 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-21 23:08:19.030984021 +0000 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-21 23:08:18.786986194 +0000 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-21 23:08:18.786986194 +0000 2024-09-21T23:13:57.951 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-21T23:13:57.952 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T23:13:58.006 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-21T23:13:58.006 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-21T23:13:58.006 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000450912 s, 1.1 MB/s 2024-09-21T23:13:58.007 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T23:13:58.057 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-21 23:08:19.498979852 +0000 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-21 23:08:19.242982132 +0000 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-21 23:08:19.242982132 +0000 2024-09-21T23:13:58.103 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-21T23:13:58.103 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T23:13:58.158 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-21T23:13:58.158 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-21T23:13:58.158 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000442426 s, 1.2 MB/s 2024-09-21T23:13:58.159 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T23:13:58.208 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-21 23:08:19.986975505 +0000 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-21 23:08:19.746977643 +0000 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-21 23:08:19.746977643 +0000 2024-09-21T23:13:58.255 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-21T23:13:58.255 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T23:13:58.310 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-21T23:13:58.310 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-21T23:13:58.310 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000430523 s, 1.2 MB/s 2024-09-21T23:13:58.311 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T23:13:58.361 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-09-21T23:13:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T23:13:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T23:13:58.407 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 936 Links: 1 2024-09-21T23:13:58.408 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T23:13:58.408 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-21 23:08:20.462971265 +0000 2024-09-21T23:13:58.408 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-21 23:08:20.210973509 +0000 2024-09-21T23:13:58.408 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-21 23:08:20.210973509 +0000 2024-09-21T23:13:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-21T23:13:58.408 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T23:13:58.461 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-21T23:13:58.461 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-21T23:13:58.461 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000300769 s, 1.7 MB/s 2024-09-21T23:13:58.462 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T23:13:58.512 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:13:58.512 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-21T23:13:58.512 INFO:tasks.ceph:Generating config... 2024-09-21T23:13:58.513 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T23:13:58.513 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T23:13:58.513 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T23:13:58.514 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T23:13:58.514 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T23:13:58.514 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T23:13:58.514 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T23:13:58.514 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T23:13:58.514 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-21T23:13:58.515 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T23:13:58.515 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T23:13:58.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T23:13:58.551 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T23:13:58.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T23:13:58.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:13:58.706 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.53'), ('mon.c', '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]'), ('mon.b', '172.21.15.144')] 2024-09-21T23:13:58.706 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.144', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T23:13:58.707 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:58.707 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T23:13:58.754 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.53 --addv c '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]' --add b 172.21.15.144 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: generated fsid 309803bf-3365-47ac-8bf6-365125beceb1 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = quincy 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 0 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:fsid 309803bf-3365-47ac-8bf6-365125beceb1 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:last_changed 2024-09-21T23:13:58.842481+0000 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-09-21T23:13:58.842481+0000 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:min_mon_release 17 (quincy) 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:election_strategy: 1 2024-09-21T23:13:58.841 INFO:teuthology.orchestra.run.smithi053.stdout:0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-09-21T23:13:58.842 INFO:teuthology.orchestra.run.smithi053.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-09-21T23:13:58.842 INFO:teuthology.orchestra.run.smithi053.stdout:2: [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] mon.c 2024-09-21T23:13:58.842 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T23:13:58.843 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T23:13:58.889 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 309803bf-3365-47ac-8bf6-365125beceb1... 2024-09-21T23:13:58.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:13:58.934 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:13:58.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-09-21T23:13:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-09-21T23:13:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-09-21T23:13:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.144" 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: mon client directed command retry = 5 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on pool no app = False 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: ms inject socket failures = 5000 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 309803bf-3365-47ac-8bf6-365125beceb1 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-09-21T23:13:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: bdev async discard = True 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: bdev enable discard = True 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-09-21T23:13:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 1/20 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 1/20 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 4/10 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: osd max markdown count = 1000 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: osd memory target = 1610612736 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-09-21T23:13:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-09-21T23:13:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:13:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-09-21T23:13:58.970 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.971 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.972 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.973 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = "172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.144" 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: mon client directed command retry = 5 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on pool no app = False 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: ms inject socket failures = 5000 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = 309803bf-3365-47ac-8bf6-365125beceb1 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2024-09-21T23:13:58.974 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug trim objects = true 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.975 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: bdev async discard = True 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: bdev enable discard = True 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2024-09-21T23:13:58.976 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 1/20 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 1/20 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 4/10 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: osd max markdown count = 1000 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: osd memory target = 1610612736 2024-09-21T23:13:58.977 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:13:58.978 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:13:58.979 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:13:58.980 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.c] 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.b] 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-09-21T23:13:58.981 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-09-21T23:13:58.982 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-09-21T23:13:58.983 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.144" 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: mon client directed command retry = 5 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on pool no app = False 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject socket failures = 5000 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 309803bf-3365-47ac-8bf6-365125beceb1 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-09-21T23:13:58.984 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-09-21T23:13:58.985 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-09-21T23:13:58.986 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: osd blocked scrub grace period = 3600 2024-09-21T23:13:58.987 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: osd max markdown count = 1000 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: osd memory target = 1610612736 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.988 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T23:13:58.989 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-09-21T23:13:58.990 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-09-21T23:13:58.991 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T23:13:58.991 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T23:13:58.991 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-09-21T23:13:58.991 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-09-21T23:13:58.991 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-09-21T23:13:58.992 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T23:13:58.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T23:13:59.084 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T23:13:59.084 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:59.084 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T23:13:59.089 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:59.089 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T23:13:59.137 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:13:59.137 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:59.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:13:59.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:13:59.201 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:13:59.201 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:13:59.249 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:13:59.249 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:13:59.249 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:13:59.249 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:13:59.272 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:13:59.272 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:13:59.319 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T23:13:59.319 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:13:59.319 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T23:13:59.319 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T23:13:59.343 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:13:59.343 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:13:59.391 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T23:13:59.391 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T23:13:59.391 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T23:13:59.435 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T23:13:59.458 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T23:13:59.458 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T23:13:59.458 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T23:13:59.494 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T23:13:59.518 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T23:13:59.518 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-21T23:13:59.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T23:13:59.532 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:13:59.533 INFO:tasks.ceph:role: osd.0 2024-09-21T23:13:59.533 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:13:59.533 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T23:13:59.590 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:13:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:13:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:13:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:13:59.590 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:13:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:13:59.590 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:13:59.591 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:13:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:13:59.591 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:13:59.613 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-21T23:13:59.616 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-21T23:13:59.616 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T23:13:59.675 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T23:13:59.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:13:59.727 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:13:59.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T23:13:59.782 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:13:59.782 INFO:tasks.ceph:role: osd.1 2024-09-21T23:13:59.782 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:13:59.782 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:13:59.837 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:13:59.861 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-21T23:13:59.863 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-21T23:13:59.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T23:13:59.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T23:13:59.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:13:59.974 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:13:59.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T23:14:00.028 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T23:14:00.029 INFO:tasks.ceph:role: osd.2 2024-09-21T23:14:00.029 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:14:00.029 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T23:14:00.086 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:14:00.086 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:14:00.086 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:14:00.086 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:14:00.087 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:14:00.087 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:14:00.087 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:14:00.087 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:14:00.087 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:14:00.087 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:14:00.110 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-21T23:14:00.112 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-21T23:14:00.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T23:14:00.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T23:14:00.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:14:00.223 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:14:00.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:00.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:00.325+0000 7f7ae9c077c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T23:14:00.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:00.325+0000 7f7ae9c077c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T23:14:00.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:00.325+0000 7f7ae9c077c0 -1 bdev(0x5557d7d41800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T23:14:00.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:00.325+0000 7f7ae9c077c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T23:14:03.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T23:14:03.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:03.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:03.581+0000 7f09dfec87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T23:14:03.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:03.581+0000 7f09dfec87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T23:14:03.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:03.581+0000 7f09dfec87c0 -1 bdev(0x55bdfaaab800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T23:14:03.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:03.585+0000 7f09dfec87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T23:14:07.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T23:14:07.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:07.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:07.593+0000 7f54cb82c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T23:14:07.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:07.593+0000 7f54cb82c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T23:14:07.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:07.593+0000 7f54cb82c7c0 -1 bdev(0x557811e2d800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T23:14:07.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:07.593+0000 7f54cb82c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T23:14:11.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T23:14:11.523 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T23:14:11.538 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:14:11.538 INFO:tasks.ceph:role: osd.3 2024-09-21T23:14:11.538 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:14:11.538 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:14:11.600 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:14:12.083 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-21T23:14:12.085 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-21T23:14:12.085 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-21T23:14:12.242 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T23:14:12.254 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:14:12.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:14:12.255 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T23:14:12.311 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:14:12.311 INFO:tasks.ceph:role: osd.4 2024-09-21T23:14:12.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:14:12.311 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T23:14:12.377 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:14:12.377 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:14:12.377 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:14:12.378 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:14:12.378 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:14:12.378 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:14:12.378 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:14:12.378 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:14:12.378 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:14:12.378 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:14:12.828 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-21T23:14:12.830 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-21T23:14:12.831 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-21T23:14:12.922 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T23:14:12.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:14:12.932 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:14:12.932 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T23:14:12.985 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T23:14:12.985 INFO:tasks.ceph:role: osd.5 2024-09-21T23:14:12.985 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:14:12.985 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T23:14:13.047 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T23:14:13.512 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-21T23:14:13.514 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-21T23:14:13.515 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-21T23:14:13.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T23:14:13.663 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T23:14:13.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:14:13.664 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:13.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:13.759+0000 7f36f5ba27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T23:14:13.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:13.763+0000 7f36f5ba27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T23:14:13.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:13.763+0000 7f36f5ba27c0 -1 bdev(0x55efca925800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T23:14:13.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:13.763+0000 7f36f5ba27c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T23:14:17.624 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T23:14:17.638 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:17.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:17.711+0000 7f464a0877c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T23:14:17.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:17.711+0000 7f464a0877c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T23:14:17.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:17.711+0000 7f464a0877c0 -1 bdev(0x56455c7c5800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T23:14:17.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:17.711+0000 7f464a0877c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T23:14:21.610 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T23:14:21.664 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:21.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:21.739+0000 7fe85a2777c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T23:14:21.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:21.739+0000 7fe85a2777c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T23:14:21.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:21.739+0000 7fe85a2777c0 -1 bdev(0x56245ef57800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T23:14:21.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:21.739+0000 7fe85a2777c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T23:14:25.158 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T23:14:25.212 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T23:14:25.212 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:14:25.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T23:14:25.229 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:14:25.229 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T23:14:25.285 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:14:25.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T23:14:25.342 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:14:25.342 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T23:14:25.355 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:14:25.355 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T23:14:25.411 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:14:25.411 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T23:14:25.466 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:14:25.466 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T23:14:25.517 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T23:14:25.518 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T23:14:25.524 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T23:14:25.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T23:14:25.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T23:14:25.561 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2024-09-21T23:14:25.561 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA4U+9minqGExAAlRD2bpb6c1nz3vwTWqjojw== 2024-09-21T23:14:25.561 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2024-09-21T23:14:25.561 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA7U+9mNGbwIhAAYRtZeslJ52KRnYcze/Q5Eg== 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA/U+9mQJSPIxAAUXARv4FHKHET8pEHK5rmNg== 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA3U+9mSSLiGRAAYwuwmHucAOp40EKHbzQ/oA== 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBFU+9mo92cLRAAwS0qLIByrLTcHQJtueeahQ== 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBJU+9mmeyzKhAAPfYC5omOLqGPXw4MokvBjQ== 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBNU+9mgzFYLBAABKUMwk1EtLtqwKnYucsEiQ== 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-09-21T23:14:25.562 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA3U+9mXp2RHRAAH1DP3wYWgn5yZOafBeJ29A== 2024-09-21T23:14:25.568 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2024-09-21T23:14:25.568 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQA4U+9minqGExAAlRD2bpb6c1nz3vwTWqjojw== 2024-09-21T23:14:25.568 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQA7U+9mNGbwIhAAYRtZeslJ52KRnYcze/Q5Eg== 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQA/U+9mQJSPIxAAUXARv4FHKHET8pEHK5rmNg== 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQA3U+9mSSLiGRAAYwuwmHucAOp40EKHbzQ/oA== 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBFU+9mo92cLRAAwS0qLIByrLTcHQJtueeahQ== 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBJU+9mmeyzKhAAPfYC5omOLqGPXw4MokvBjQ== 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBNU+9mgzFYLBAABKUMwk1EtLtqwKnYucsEiQ== 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-09-21T23:14:25.569 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQA3U+9mXp2RHRAAH1DP3wYWgn5yZOafBeJ29A== 2024-09-21T23:14:25.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.606 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.661 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.746 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:14:25.830 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T23:14:25.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.914 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:25.961 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:26.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:26.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T23:14:26.118 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:14:26.130 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T23:14:26.205 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T23:14:26.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T23:14:26.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:14:26.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T23:14:26.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T23:14:26.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:14:26.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T23:14:26.474 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T23:14:26.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T23:14:26.585 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T23:14:26.596 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:26.598 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T23:14:26.642 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T23:14:26.642 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T23:14:26.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T23:14:26.646 INFO:tasks.ceph.mon.a:Started 2024-09-21T23:14:26.646 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T23:14:26.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T23:14:26.649 INFO:tasks.ceph.mon.c:Started 2024-09-21T23:14:26.649 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T23:14:26.649 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T23:14:26.688 INFO:tasks.ceph.mon.b:Started 2024-09-21T23:14:26.688 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T23:14:26.688 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T23:14:26.688 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T23:14:26.690 INFO:tasks.ceph.mgr.x:Started 2024-09-21T23:14:26.690 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T23:14:26.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T23:14:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T23:14:32.277+0000 7f6bc7fff640 0 --2- 172.21.15.53:0/2465664593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb40052f0 0x7f6bb400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T23:14:32.337 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:32.331+0000 7fc4580fc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T23:14:32.361 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2024-09-21T23:14:32.381 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T23:14:32.381 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T23:14:32.381 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:14:32.381 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T23:14:32.393 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:14:32.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T23:14:32.448 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T23:14:32.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T23:14:32.505 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:14:32.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T23:14:32.518 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:14:32.518 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T23:14:32.573 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-21T23:14:32.574 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T23:14:32.626 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new a24df615-4443-4f27-888f-d1e4e7affbb3 0 2024-09-21T23:14:32.869 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:32.863+0000 7fc4580fc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T23:14:32.887 INFO:teuthology.orchestra.run.smithi144.stdout:0 2024-09-21T23:14:32.905 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new d6765c2a-98b2-4a2b-8d8c-f3963575eb64 1 2024-09-21T23:14:33.099 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:33.095+0000 7fc4580fc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T23:14:33.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-21T23:14:33.223+0000 7fc3e8e0e640 0 --2- 172.21.15.144:0/2529410936 >> v2:172.21.15.144:3300/0 conn(0x7fc3e40a09e0 0x7fc3e40a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T23:14:33.314 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-09-21T23:14:33.332 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 9753272a-58a9-4366-87d7-550c49134023 2 2024-09-21T23:14:33.535 INFO:teuthology.orchestra.run.smithi144.stdout:2 2024-09-21T23:14:33.551 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 24d987f9-f2cd-4ccd-9dc8-fea64467d92d 3 2024-09-21T23:14:33.695 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:33.691+0000 7fc4580fc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T23:14:33.749 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:33.743+0000 7fc4580fc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T23:14:33.755 INFO:teuthology.orchestra.run.smithi144.stdout:3 2024-09-21T23:14:33.771 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new c1c30a49-7f26-4264-a1a9-160dc552953d 4 2024-09-21T23:14:33.805 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:33.799+0000 7fc4580fc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T23:14:33.885 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:33.879+0000 7fc4580fc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T23:14:33.939 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:33.935+0000 7fc4580fc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T23:14:33.976 INFO:teuthology.orchestra.run.smithi144.stdout:4 2024-09-21T23:14:33.992 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new f24e9341-be4e-4be2-b180-7ebd3f705e9b 5 2024-09-21T23:14:34.020 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:34.015+0000 7fc4580fc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T23:14:34.075 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:34.071+0000 7fc4580fc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T23:14:34.219 INFO:teuthology.orchestra.run.smithi144.stdout:5 2024-09-21T23:14:34.233 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:34.227+0000 7fc4580fc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T23:14:34.236 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T23:14:34.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T23:14:34.238 INFO:tasks.ceph.osd.0:Started 2024-09-21T23:14:34.238 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T23:14:34.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T23:14:34.241 INFO:tasks.ceph.osd.1:Started 2024-09-21T23:14:34.241 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T23:14:34.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T23:14:34.245 INFO:tasks.ceph.osd.2:Started 2024-09-21T23:14:34.245 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T23:14:34.245 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T23:14:34.247 INFO:tasks.ceph.osd.3:Started 2024-09-21T23:14:34.247 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T23:14:34.247 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T23:14:34.249 INFO:tasks.ceph.osd.4:Started 2024-09-21T23:14:34.249 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T23:14:34.249 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T23:14:34.251 INFO:tasks.ceph.osd.5:Started 2024-09-21T23:14:34.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:14:34.301 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:34.295+0000 7fc4580fc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T23:14:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:34.448 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":8,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:14:34.216116+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:14:34.216116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:14:34.466 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T23:14:34.466 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T23:14:34.873 INFO:tasks.ceph.mgr.x.smithi144.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T23:14:34.883 INFO:tasks.ceph.mgr.x.smithi144.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T23:14:34.883 INFO:tasks.ceph.mgr.x.smithi144.stderr: from numpy import show_config as show_numpy_config 2024-09-21T23:14:34.926 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:34.919+0000 7fc4580fc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T23:14:35.072 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:14:35.098 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:35.091+0000 7fc4580fc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:14:35.269 INFO:teuthology.misc.health.smithi053.stdout: 2024-09-21T23:14:35.269 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":8,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:14:34.216116+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:14:34.216116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:14:35.282 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:35.275+0000 7fc4580fc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T23:14:35.286 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-21T23:14:35.352 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:35.347+0000 7fc4580fc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T23:14:35.500 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:35.495+0000 7fc4580fc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T23:14:35.636 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:14:35.631+0000 7f05608e27c0 -1 Falling back to public interface 2024-09-21T23:14:35.879 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:14:35.876+0000 7f471c77b7c0 -1 Falling back to public interface 2024-09-21T23:14:35.890 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:14:35.888+0000 7f7600e5f7c0 -1 Falling back to public interface 2024-09-21T23:14:36.099 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:14:36.096+0000 7f32629627c0 -1 Falling back to public interface 2024-09-21T23:14:36.123 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:36.119+0000 7fc4580fc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T23:14:36.130 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:14:36.123+0000 7f9fc111a7c0 -1 Falling back to public interface 2024-09-21T23:14:36.130 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:14:36.123+0000 7f224653a7c0 -1 Falling back to public interface 2024-09-21T23:14:36.201 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:36.195+0000 7fc4580fc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T23:14:36.383 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:36.375+0000 7fc4580fc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T23:14:36.456 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:36.451+0000 7fc4580fc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T23:14:36.748 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:36.743+0000 7fc4580fc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T23:14:36.810 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:36.803+0000 7fc4580fc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T23:14:37.850 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:37.843+0000 7fc3fec30640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T23:14:37.851 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-21T23:14:37.847+0000 7fc3fec30640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T23:14:40.089 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:14:40.088+0000 7f7600e5f7c0 -1 osd.1 0 log_to_monitors true 2024-09-21T23:14:40.253 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:14:40.247+0000 7f9fc111a7c0 -1 osd.3 0 log_to_monitors true 2024-09-21T23:14:40.292 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:14:40.288+0000 7f471c77b7c0 -1 osd.2 0 log_to_monitors true 2024-09-21T23:14:40.403 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:14:40.395+0000 7f05608e27c0 -1 osd.5 0 log_to_monitors true 2024-09-21T23:14:40.493 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:14:40.492+0000 7f32629627c0 -1 osd.0 0 log_to_monitors true 2024-09-21T23:14:40.852 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:14:40.847+0000 7f224653a7c0 -1 osd.4 0 log_to_monitors true 2024-09-21T23:14:41.892 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:14:42.295 INFO:teuthology.misc.health.smithi053.stdout: 2024-09-21T23:14:42.295 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":10,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:14:41.848760+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T23:14:34.216116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:14:42.319 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-21T23:14:43.180 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:14:43.176+0000 7f325e0e8640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T23:14:43.186 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:14:43.184+0000 7f75fc5e5640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T23:14:43.196 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:14:43.192+0000 7f4718f36640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T23:14:43.209 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:14:43.208+0000 7f325672f640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:14:43.217 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:14:43.216+0000 7f75f4c2c640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:14:43.237 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:14:43.236+0000 7f4710548640 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:14:43.521 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:14:43.515+0000 7f055c068640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T23:14:43.552 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:14:43.547+0000 7f05546af640 -1 osd.5 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:14:43.575 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:14:43.567+0000 7f9fbd8d5640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T23:14:43.615 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:14:43.611+0000 7f9fb4ee7640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:14:44.005 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:14:43.999+0000 7f2241cc0640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T23:14:44.032 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:14:44.027+0000 7f223a307640 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:14:48.926 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T23:14:49.233 INFO:teuthology.misc.health.smithi053.stdout: 2024-09-21T23:14:49.234 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":15,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:14:47.212498+0000","last_up_change":"2024-09-21T23:14:44.204301+0000","last_in_change":"2024-09-21T23:14:34.216116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:14:44.926374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6817","nonce":91058490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6819","nonce":91058490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6823","nonce":91058490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6821","nonce":91058490}]},"public_addr":"172.21.15.53:6817/91058490","cluster_addr":"172.21.15.53:6819/91058490","heartbeat_back_addr":"172.21.15.53:6823/91058490","heartbeat_front_addr":"172.21.15.53:6821/91058490","state":["exists","up"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6809","nonce":2712042978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6811","nonce":2712042978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6815","nonce":2712042978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6813","nonce":2712042978}]},"public_addr":"172.21.15.53:6809/2712042978","cluster_addr":"172.21.15.53:6811/2712042978","heartbeat_back_addr":"172.21.15.53:6815/2712042978","heartbeat_front_addr":"172.21.15.53:6813/2712042978","state":["exists","up"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6801","nonce":1166517858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6803","nonce":1166517858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6807","nonce":1166517858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6805","nonce":1166517858}]},"public_addr":"172.21.15.53:6801/1166517858","cluster_addr":"172.21.15.53:6803/1166517858","heartbeat_back_addr":"172.21.15.53:6807/1166517858","heartbeat_front_addr":"172.21.15.53:6805/1166517858","state":["exists","up"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6811","nonce":1039392120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6816","nonce":1039392120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6823","nonce":1039392120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6821","nonce":1039392120}]},"public_addr":"172.21.15.144:6811/1039392120","cluster_addr":"172.21.15.144:6816/1039392120","heartbeat_back_addr":"172.21.15.144:6823/1039392120","heartbeat_front_addr":"172.21.15.144:6821/1039392120","state":["exists","up"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6810","nonce":2764095640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6813","nonce":2764095640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6820","nonce":2764095640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6817","nonce":2764095640}]},"public_addr":"172.21.15.144:6810/2764095640","cluster_addr":"172.21.15.144:6813/2764095640","heartbeat_back_addr":"172.21.15.144:6820/2764095640","heartbeat_front_addr":"172.21.15.144:6817/2764095640","state":["exists","up"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6801","nonce":1732973431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6803","nonce":1732973431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6807","nonce":1732973431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6805","nonce":1732973431}]},"public_addr":"172.21.15.144:6801/1732973431","cluster_addr":"172.21.15.144:6803/1732973431","heartbeat_back_addr":"172.21.15.144:6807/1732973431","heartbeat_front_addr":"172.21.15.144:6805/1732973431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.468749+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.044837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.310970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.241402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.877818+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.434052+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:14:49.251 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-21T23:14:49.251 INFO:tasks.ceph:Creating RBD pool 2024-09-21T23:14:49.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T23:14:50.240 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'rbd' created 2024-09-21T23:14:50.260 DEBUG:teuthology.orchestra.run.smithi053:> rbd --cluster ceph pool init rbd 2024-09-21T23:14:53.280 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T23:14:53.281 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T23:14:53.281 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T23:14:53.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T23:14:53.281 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T23:14:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:53.660 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3061128477},{"type":"v1","addr":"172.21.15.144:6825","nonce":3061128477}]},"active_addr":"172.21.15.144:6825/3061128477","active_change":"2024-09-21T23:14:36.831621+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":2818967946}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":282926801}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":2096930827}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":3437650986}]}]} 2024-09-21T23:14:53.662 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T23:14:53.662 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T23:14:53.662 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:14:53.965 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:53.965 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":19,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:14:53.253655+0000","last_up_change":"2024-09-21T23:14:44.204301+0000","last_in_change":"2024-09-21T23:14:34.216116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:14:44.926374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:14:49.598603+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6817","nonce":91058490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6819","nonce":91058490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6823","nonce":91058490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6821","nonce":91058490}]},"public_addr":"172.21.15.53:6817/91058490","cluster_addr":"172.21.15.53:6819/91058490","heartbeat_back_addr":"172.21.15.53:6823/91058490","heartbeat_front_addr":"172.21.15.53:6821/91058490","state":["exists","up"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6809","nonce":2712042978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6811","nonce":2712042978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6815","nonce":2712042978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6813","nonce":2712042978}]},"public_addr":"172.21.15.53:6809/2712042978","cluster_addr":"172.21.15.53:6811/2712042978","heartbeat_back_addr":"172.21.15.53:6815/2712042978","heartbeat_front_addr":"172.21.15.53:6813/2712042978","state":["exists","up"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6801","nonce":1166517858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6803","nonce":1166517858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6807","nonce":1166517858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6805","nonce":1166517858}]},"public_addr":"172.21.15.53:6801/1166517858","cluster_addr":"172.21.15.53:6803/1166517858","heartbeat_back_addr":"172.21.15.53:6807/1166517858","heartbeat_front_addr":"172.21.15.53:6805/1166517858","state":["exists","up"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6811","nonce":1039392120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6816","nonce":1039392120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6823","nonce":1039392120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6821","nonce":1039392120}]},"public_addr":"172.21.15.144:6811/1039392120","cluster_addr":"172.21.15.144:6816/1039392120","heartbeat_back_addr":"172.21.15.144:6823/1039392120","heartbeat_front_addr":"172.21.15.144:6821/1039392120","state":["exists","up"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6810","nonce":2764095640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6813","nonce":2764095640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6820","nonce":2764095640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6817","nonce":2764095640}]},"public_addr":"172.21.15.144:6810/2764095640","cluster_addr":"172.21.15.144:6813/2764095640","heartbeat_back_addr":"172.21.15.144:6820/2764095640","heartbeat_front_addr":"172.21.15.144:6817/2764095640","state":["exists","up"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6801","nonce":1732973431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6803","nonce":1732973431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6807","nonce":1732973431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6805","nonce":1732973431}]},"public_addr":"172.21.15.144:6801/1732973431","cluster_addr":"172.21.15.144:6803/1732973431","heartbeat_back_addr":"172.21.15.144:6807/1732973431","heartbeat_front_addr":"172.21.15.144:6805/1732973431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.468749+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.044837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.310970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.241402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.877818+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.434052+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:14:53.983 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T23:14:53.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:14:54.266 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:54.266 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":19,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:14:53.253655+0000","last_up_change":"2024-09-21T23:14:44.204301+0000","last_in_change":"2024-09-21T23:14:34.216116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:14:44.926374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:14:49.598603+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6817","nonce":91058490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6819","nonce":91058490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6823","nonce":91058490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":91058490},{"type":"v1","addr":"172.21.15.53:6821","nonce":91058490}]},"public_addr":"172.21.15.53:6817/91058490","cluster_addr":"172.21.15.53:6819/91058490","heartbeat_back_addr":"172.21.15.53:6823/91058490","heartbeat_front_addr":"172.21.15.53:6821/91058490","state":["exists","up"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6809","nonce":2712042978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6811","nonce":2712042978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6815","nonce":2712042978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6813","nonce":2712042978}]},"public_addr":"172.21.15.53:6809/2712042978","cluster_addr":"172.21.15.53:6811/2712042978","heartbeat_back_addr":"172.21.15.53:6815/2712042978","heartbeat_front_addr":"172.21.15.53:6813/2712042978","state":["exists","up"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6801","nonce":1166517858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6803","nonce":1166517858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6807","nonce":1166517858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6805","nonce":1166517858}]},"public_addr":"172.21.15.53:6801/1166517858","cluster_addr":"172.21.15.53:6803/1166517858","heartbeat_back_addr":"172.21.15.53:6807/1166517858","heartbeat_front_addr":"172.21.15.53:6805/1166517858","state":["exists","up"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6811","nonce":1039392120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6816","nonce":1039392120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6823","nonce":1039392120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6821","nonce":1039392120}]},"public_addr":"172.21.15.144:6811/1039392120","cluster_addr":"172.21.15.144:6816/1039392120","heartbeat_back_addr":"172.21.15.144:6823/1039392120","heartbeat_front_addr":"172.21.15.144:6821/1039392120","state":["exists","up"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6810","nonce":2764095640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6813","nonce":2764095640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6820","nonce":2764095640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6817","nonce":2764095640}]},"public_addr":"172.21.15.144:6810/2764095640","cluster_addr":"172.21.15.144:6813/2764095640","heartbeat_back_addr":"172.21.15.144:6820/2764095640","heartbeat_front_addr":"172.21.15.144:6817/2764095640","state":["exists","up"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6801","nonce":1732973431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6803","nonce":1732973431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6807","nonce":1732973431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6805","nonce":1732973431}]},"public_addr":"172.21.15.144:6801/1732973431","cluster_addr":"172.21.15.144:6803/1732973431","heartbeat_back_addr":"172.21.15.144:6807/1732973431","heartbeat_front_addr":"172.21.15.144:6805/1732973431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.468749+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.044837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.310970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.241402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.877818+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.434052+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:14:54.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T23:14:54.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T23:14:54.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T23:14:54.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T23:14:54.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T23:14:54.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T23:14:54.461 INFO:teuthology.orchestra.run.smithi053.stdout:51539607558 2024-09-21T23:14:54.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:14:54.478 INFO:teuthology.orchestra.run.smithi053.stdout:51539607558 2024-09-21T23:14:54.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:14:54.513 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:54.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:14:54.522 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:54.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:14:54.528 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:54.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:14:54.532 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:54.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:14:54.843 INFO:teuthology.orchestra.run.smithi053.stdout:51539607556 2024-09-21T23:14:54.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.3 2024-09-21T23:14:54.893 INFO:teuthology.orchestra.run.smithi053.stdout:51539607556 2024-09-21T23:14:54.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.2 2024-09-21T23:14:54.935 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:54.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.1 2024-09-21T23:14:54.956 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:54.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.0 2024-09-21T23:14:54.997 INFO:teuthology.orchestra.run.smithi053.stdout:51539607556 2024-09-21T23:14:55.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.5 2024-09-21T23:14:55.032 INFO:teuthology.orchestra.run.smithi053.stdout:51539607556 2024-09-21T23:14:55.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.4 2024-09-21T23:14:55.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:14:55.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:14:55.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:14:55.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:14:56.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:14:56.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:14:56.248 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:56.266 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:56.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.2 2024-09-21T23:14:56.270 DEBUG:teuthology.parallel:result is None 2024-09-21T23:14:56.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.3 2024-09-21T23:14:56.291 DEBUG:teuthology.parallel:result is None 2024-09-21T23:14:56.367 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:56.375 INFO:teuthology.orchestra.run.smithi053.stdout:51539607558 2024-09-21T23:14:56.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.5 2024-09-21T23:14:56.386 DEBUG:teuthology.parallel:result is None 2024-09-21T23:14:56.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.1 2024-09-21T23:14:56.394 DEBUG:teuthology.parallel:result is None 2024-09-21T23:14:56.414 INFO:teuthology.orchestra.run.smithi053.stdout:51539607558 2024-09-21T23:14:56.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.0 2024-09-21T23:14:56.431 DEBUG:teuthology.parallel:result is None 2024-09-21T23:14:56.437 INFO:teuthology.orchestra.run.smithi053.stdout:51539607557 2024-09-21T23:14:56.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.4 2024-09-21T23:14:56.454 DEBUG:teuthology.parallel:result is None 2024-09-21T23:14:56.454 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:14:56.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:14:56.776 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:14:56.792 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-21T23:14:56.731951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":176900,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566054140,"statfs":{"total":579820584960,"available":579639439360,"internally_reserved":0,"allocated":2277376,"data_stored":1539058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.891179"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262843+0000","last_change":"2024-09-21T23:14:53.262843+0000","last_active":"2024-09-21T23:14:53.262843+0000","last_peered":"2024-09-21T23:14:53.262843+0000","last_clean":"2024-09-21T23:14:53.262843+0000","last_became_active":"2024-09-21T23:14:51.360795+0000","last_became_peered":"2024-09-21T23:14:51.360795+0000","last_unstale":"2024-09-21T23:14:53.262843+0000","last_undegraded":"2024-09-21T23:14:53.262843+0000","last_fullsized":"2024-09-21T23:14:53.262843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:10.419450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016301600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.274401+0000","last_change":"2024-09-21T23:14:53.274401+0000","last_active":"2024-09-21T23:14:53.274401+0000","last_peered":"2024-09-21T23:14:53.274401+0000","last_clean":"2024-09-21T23:14:53.274401+0000","last_became_active":"2024-09-21T23:14:51.253820+0000","last_became_peered":"2024-09-21T23:14:51.253820+0000","last_unstale":"2024-09-21T23:14:53.274401+0000","last_undegraded":"2024-09-21T23:14:53.274401+0000","last_fullsized":"2024-09-21T23:14:53.274401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:57.046270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262402+0000","last_change":"2024-09-21T23:14:53.262402+0000","last_active":"2024-09-21T23:14:53.262402+0000","last_peered":"2024-09-21T23:14:53.262402+0000","last_clean":"2024-09-21T23:14:53.262402+0000","last_became_active":"2024-09-21T23:14:51.500246+0000","last_became_peered":"2024-09-21T23:14:51.500246+0000","last_unstale":"2024-09-21T23:14:53.262402+0000","last_undegraded":"2024-09-21T23:14:53.262402+0000","last_fullsized":"2024-09-21T23:14:53.262402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:33:42.165228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012328759999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.275066+0000","last_change":"2024-09-21T23:14:53.275066+0000","last_active":"2024-09-21T23:14:53.275066+0000","last_peered":"2024-09-21T23:14:53.275066+0000","last_clean":"2024-09-21T23:14:53.275066+0000","last_became_active":"2024-09-21T23:14:51.503577+0000","last_became_peered":"2024-09-21T23:14:51.503577+0000","last_unstale":"2024-09-21T23:14:53.275066+0000","last_undegraded":"2024-09-21T23:14:53.275066+0000","last_fullsized":"2024-09-21T23:14:53.275066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:34:53.168488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.269144+0000","last_change":"2024-09-21T23:14:53.267881+0000","last_active":"2024-09-21T23:14:53.269144+0000","last_peered":"2024-09-21T23:14:53.269144+0000","last_clean":"2024-09-21T23:14:53.269144+0000","last_became_active":"2024-09-21T23:14:51.255582+0000","last_became_peered":"2024-09-21T23:14:51.255582+0000","last_unstale":"2024-09-21T23:14:53.269144+0000","last_undegraded":"2024-09-21T23:14:53.269144+0000","last_fullsized":"2024-09-21T23:14:53.269144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:54.091377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:54.112260+0000","last_change":"2024-09-21T23:14:54.112260+0000","last_active":"2024-09-21T23:14:54.112260+0000","last_peered":"2024-09-21T23:14:54.112260+0000","last_clean":"2024-09-21T23:14:54.112260+0000","last_became_active":"2024-09-21T23:14:51.257663+0000","last_became_peered":"2024-09-21T23:14:51.257663+0000","last_unstale":"2024-09-21T23:14:54.112260+0000","last_undegraded":"2024-09-21T23:14:54.112260+0000","last_fullsized":"2024-09-21T23:14:54.112260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:20.628422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262129+0000","last_change":"2024-09-21T23:14:53.262129+0000","last_active":"2024-09-21T23:14:53.262129+0000","last_peered":"2024-09-21T23:14:53.262129+0000","last_clean":"2024-09-21T23:14:53.262129+0000","last_became_active":"2024-09-21T23:14:51.249890+0000","last_became_peered":"2024-09-21T23:14:51.249890+0000","last_unstale":"2024-09-21T23:14:53.262129+0000","last_undegraded":"2024-09-21T23:14:53.262129+0000","last_fullsized":"2024-09-21T23:14:53.262129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:51:33.714563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.266479+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:14:53.266479+0000","last_peered":"2024-09-21T23:14:53.266479+0000","last_clean":"2024-09-21T23:14:53.266479+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:14:53.266479+0000","last_undegraded":"2024-09-21T23:14:53.266479+0000","last_fullsized":"2024-09-21T23:14:53.266479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.260975+0000","last_change":"2024-09-21T23:14:46.402802+0000","last_active":"2024-09-21T23:14:53.260975+0000","last_peered":"2024-09-21T23:14:53.260975+0000","last_clean":"2024-09-21T23:14:53.260975+0000","last_became_active":"2024-09-21T23:14:46.400426+0000","last_became_peered":"2024-09-21T23:14:46.400426+0000","last_unstale":"2024-09-21T23:14:53.260975+0000","last_undegraded":"2024-09-21T23:14:53.260975+0000","last_fullsized":"2024-09-21T23:14:53.260975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:34.310611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:14:56.793 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:14:57.088 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:14:57.104 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T23:14:56.842488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":176900,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566054140,"statfs":{"total":579820584960,"available":579639439360,"internally_reserved":0,"allocated":2277376,"data_stored":1539058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.599200"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262843+0000","last_change":"2024-09-21T23:14:53.262843+0000","last_active":"2024-09-21T23:14:53.262843+0000","last_peered":"2024-09-21T23:14:53.262843+0000","last_clean":"2024-09-21T23:14:53.262843+0000","last_became_active":"2024-09-21T23:14:51.360795+0000","last_became_peered":"2024-09-21T23:14:51.360795+0000","last_unstale":"2024-09-21T23:14:53.262843+0000","last_undegraded":"2024-09-21T23:14:53.262843+0000","last_fullsized":"2024-09-21T23:14:53.262843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:10.419450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016301600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.274401+0000","last_change":"2024-09-21T23:14:53.274401+0000","last_active":"2024-09-21T23:14:53.274401+0000","last_peered":"2024-09-21T23:14:53.274401+0000","last_clean":"2024-09-21T23:14:53.274401+0000","last_became_active":"2024-09-21T23:14:51.253820+0000","last_became_peered":"2024-09-21T23:14:51.253820+0000","last_unstale":"2024-09-21T23:14:53.274401+0000","last_undegraded":"2024-09-21T23:14:53.274401+0000","last_fullsized":"2024-09-21T23:14:53.274401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:57.046270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262402+0000","last_change":"2024-09-21T23:14:53.262402+0000","last_active":"2024-09-21T23:14:53.262402+0000","last_peered":"2024-09-21T23:14:53.262402+0000","last_clean":"2024-09-21T23:14:53.262402+0000","last_became_active":"2024-09-21T23:14:51.500246+0000","last_became_peered":"2024-09-21T23:14:51.500246+0000","last_unstale":"2024-09-21T23:14:53.262402+0000","last_undegraded":"2024-09-21T23:14:53.262402+0000","last_fullsized":"2024-09-21T23:14:53.262402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:33:42.165228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012328759999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.275066+0000","last_change":"2024-09-21T23:14:53.275066+0000","last_active":"2024-09-21T23:14:53.275066+0000","last_peered":"2024-09-21T23:14:53.275066+0000","last_clean":"2024-09-21T23:14:53.275066+0000","last_became_active":"2024-09-21T23:14:51.503577+0000","last_became_peered":"2024-09-21T23:14:51.503577+0000","last_unstale":"2024-09-21T23:14:53.275066+0000","last_undegraded":"2024-09-21T23:14:53.275066+0000","last_fullsized":"2024-09-21T23:14:53.275066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:34:53.168488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.269144+0000","last_change":"2024-09-21T23:14:53.267881+0000","last_active":"2024-09-21T23:14:53.269144+0000","last_peered":"2024-09-21T23:14:53.269144+0000","last_clean":"2024-09-21T23:14:53.269144+0000","last_became_active":"2024-09-21T23:14:51.255582+0000","last_became_peered":"2024-09-21T23:14:51.255582+0000","last_unstale":"2024-09-21T23:14:53.269144+0000","last_undegraded":"2024-09-21T23:14:53.269144+0000","last_fullsized":"2024-09-21T23:14:53.269144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:54.091377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:54.112260+0000","last_change":"2024-09-21T23:14:54.112260+0000","last_active":"2024-09-21T23:14:54.112260+0000","last_peered":"2024-09-21T23:14:54.112260+0000","last_clean":"2024-09-21T23:14:54.112260+0000","last_became_active":"2024-09-21T23:14:51.257663+0000","last_became_peered":"2024-09-21T23:14:51.257663+0000","last_unstale":"2024-09-21T23:14:54.112260+0000","last_undegraded":"2024-09-21T23:14:54.112260+0000","last_fullsized":"2024-09-21T23:14:54.112260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:20.628422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262129+0000","last_change":"2024-09-21T23:14:53.262129+0000","last_active":"2024-09-21T23:14:53.262129+0000","last_peered":"2024-09-21T23:14:53.262129+0000","last_clean":"2024-09-21T23:14:53.262129+0000","last_became_active":"2024-09-21T23:14:51.249890+0000","last_became_peered":"2024-09-21T23:14:51.249890+0000","last_unstale":"2024-09-21T23:14:53.262129+0000","last_undegraded":"2024-09-21T23:14:53.262129+0000","last_fullsized":"2024-09-21T23:14:53.262129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:51:33.714563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.266479+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:14:53.266479+0000","last_peered":"2024-09-21T23:14:53.266479+0000","last_clean":"2024-09-21T23:14:53.266479+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:14:53.266479+0000","last_undegraded":"2024-09-21T23:14:53.266479+0000","last_fullsized":"2024-09-21T23:14:53.266479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.260975+0000","last_change":"2024-09-21T23:14:46.402802+0000","last_active":"2024-09-21T23:14:53.260975+0000","last_peered":"2024-09-21T23:14:53.260975+0000","last_clean":"2024-09-21T23:14:53.260975+0000","last_became_active":"2024-09-21T23:14:46.400426+0000","last_became_peered":"2024-09-21T23:14:46.400426+0000","last_unstale":"2024-09-21T23:14:53.260975+0000","last_undegraded":"2024-09-21T23:14:53.260975+0000","last_fullsized":"2024-09-21T23:14:53.260975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:34.310611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:14:57.105 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:14:57.105 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T23:14:57.105 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T23:14:57.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T23:14:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:57.434 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T23:14:57.453 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T23:14:57.454 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T23:14:57.472 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T23:14:57.472 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T23:14:57.472 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T23:14:57.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T23:14:57.799 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T23:14:57.799 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:14:57.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:14:58.098 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:58.099 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:14:58.117 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T23:14:56.842488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":176900,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566054140,"statfs":{"total":579820584960,"available":579639439360,"internally_reserved":0,"allocated":2277376,"data_stored":1539058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.599200"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262843+0000","last_change":"2024-09-21T23:14:53.262843+0000","last_active":"2024-09-21T23:14:53.262843+0000","last_peered":"2024-09-21T23:14:53.262843+0000","last_clean":"2024-09-21T23:14:53.262843+0000","last_became_active":"2024-09-21T23:14:51.360795+0000","last_became_peered":"2024-09-21T23:14:51.360795+0000","last_unstale":"2024-09-21T23:14:53.262843+0000","last_undegraded":"2024-09-21T23:14:53.262843+0000","last_fullsized":"2024-09-21T23:14:53.262843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:10.419450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016301600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.274401+0000","last_change":"2024-09-21T23:14:53.274401+0000","last_active":"2024-09-21T23:14:53.274401+0000","last_peered":"2024-09-21T23:14:53.274401+0000","last_clean":"2024-09-21T23:14:53.274401+0000","last_became_active":"2024-09-21T23:14:51.253820+0000","last_became_peered":"2024-09-21T23:14:51.253820+0000","last_unstale":"2024-09-21T23:14:53.274401+0000","last_undegraded":"2024-09-21T23:14:53.274401+0000","last_fullsized":"2024-09-21T23:14:53.274401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:57.046270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262402+0000","last_change":"2024-09-21T23:14:53.262402+0000","last_active":"2024-09-21T23:14:53.262402+0000","last_peered":"2024-09-21T23:14:53.262402+0000","last_clean":"2024-09-21T23:14:53.262402+0000","last_became_active":"2024-09-21T23:14:51.500246+0000","last_became_peered":"2024-09-21T23:14:51.500246+0000","last_unstale":"2024-09-21T23:14:53.262402+0000","last_undegraded":"2024-09-21T23:14:53.262402+0000","last_fullsized":"2024-09-21T23:14:53.262402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:33:42.165228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012328759999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.275066+0000","last_change":"2024-09-21T23:14:53.275066+0000","last_active":"2024-09-21T23:14:53.275066+0000","last_peered":"2024-09-21T23:14:53.275066+0000","last_clean":"2024-09-21T23:14:53.275066+0000","last_became_active":"2024-09-21T23:14:51.503577+0000","last_became_peered":"2024-09-21T23:14:51.503577+0000","last_unstale":"2024-09-21T23:14:53.275066+0000","last_undegraded":"2024-09-21T23:14:53.275066+0000","last_fullsized":"2024-09-21T23:14:53.275066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:34:53.168488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.269144+0000","last_change":"2024-09-21T23:14:53.267881+0000","last_active":"2024-09-21T23:14:53.269144+0000","last_peered":"2024-09-21T23:14:53.269144+0000","last_clean":"2024-09-21T23:14:53.269144+0000","last_became_active":"2024-09-21T23:14:51.255582+0000","last_became_peered":"2024-09-21T23:14:51.255582+0000","last_unstale":"2024-09-21T23:14:53.269144+0000","last_undegraded":"2024-09-21T23:14:53.269144+0000","last_fullsized":"2024-09-21T23:14:53.269144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:54.091377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:54.112260+0000","last_change":"2024-09-21T23:14:54.112260+0000","last_active":"2024-09-21T23:14:54.112260+0000","last_peered":"2024-09-21T23:14:54.112260+0000","last_clean":"2024-09-21T23:14:54.112260+0000","last_became_active":"2024-09-21T23:14:51.257663+0000","last_became_peered":"2024-09-21T23:14:51.257663+0000","last_unstale":"2024-09-21T23:14:54.112260+0000","last_undegraded":"2024-09-21T23:14:54.112260+0000","last_fullsized":"2024-09-21T23:14:54.112260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:20.628422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262129+0000","last_change":"2024-09-21T23:14:53.262129+0000","last_active":"2024-09-21T23:14:53.262129+0000","last_peered":"2024-09-21T23:14:53.262129+0000","last_clean":"2024-09-21T23:14:53.262129+0000","last_became_active":"2024-09-21T23:14:51.249890+0000","last_became_peered":"2024-09-21T23:14:51.249890+0000","last_unstale":"2024-09-21T23:14:53.262129+0000","last_undegraded":"2024-09-21T23:14:53.262129+0000","last_fullsized":"2024-09-21T23:14:53.262129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:51:33.714563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.266479+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:14:53.266479+0000","last_peered":"2024-09-21T23:14:53.266479+0000","last_clean":"2024-09-21T23:14:53.266479+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:14:53.266479+0000","last_undegraded":"2024-09-21T23:14:53.266479+0000","last_fullsized":"2024-09-21T23:14:53.266479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.260975+0000","last_change":"2024-09-21T23:14:46.402802+0000","last_active":"2024-09-21T23:14:53.260975+0000","last_peered":"2024-09-21T23:14:53.260975+0000","last_clean":"2024-09-21T23:14:53.260975+0000","last_became_active":"2024-09-21T23:14:46.400426+0000","last_became_peered":"2024-09-21T23:14:46.400426+0000","last_unstale":"2024-09-21T23:14:53.260975+0000","last_undegraded":"2024-09-21T23:14:53.260975+0000","last_fullsized":"2024-09-21T23:14:53.260975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:34.310611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:14:58.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:14:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:58.417 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:14:58.435 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T23:14:56.842488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":176900,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566054140,"statfs":{"total":579820584960,"available":579639439360,"internally_reserved":0,"allocated":2277376,"data_stored":1539058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.599200"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262843+0000","last_change":"2024-09-21T23:14:53.262843+0000","last_active":"2024-09-21T23:14:53.262843+0000","last_peered":"2024-09-21T23:14:53.262843+0000","last_clean":"2024-09-21T23:14:53.262843+0000","last_became_active":"2024-09-21T23:14:51.360795+0000","last_became_peered":"2024-09-21T23:14:51.360795+0000","last_unstale":"2024-09-21T23:14:53.262843+0000","last_undegraded":"2024-09-21T23:14:53.262843+0000","last_fullsized":"2024-09-21T23:14:53.262843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:10.419450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016301600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.274401+0000","last_change":"2024-09-21T23:14:53.274401+0000","last_active":"2024-09-21T23:14:53.274401+0000","last_peered":"2024-09-21T23:14:53.274401+0000","last_clean":"2024-09-21T23:14:53.274401+0000","last_became_active":"2024-09-21T23:14:51.253820+0000","last_became_peered":"2024-09-21T23:14:51.253820+0000","last_unstale":"2024-09-21T23:14:53.274401+0000","last_undegraded":"2024-09-21T23:14:53.274401+0000","last_fullsized":"2024-09-21T23:14:53.274401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:57.046270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262402+0000","last_change":"2024-09-21T23:14:53.262402+0000","last_active":"2024-09-21T23:14:53.262402+0000","last_peered":"2024-09-21T23:14:53.262402+0000","last_clean":"2024-09-21T23:14:53.262402+0000","last_became_active":"2024-09-21T23:14:51.500246+0000","last_became_peered":"2024-09-21T23:14:51.500246+0000","last_unstale":"2024-09-21T23:14:53.262402+0000","last_undegraded":"2024-09-21T23:14:53.262402+0000","last_fullsized":"2024-09-21T23:14:53.262402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:33:42.165228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012328759999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.275066+0000","last_change":"2024-09-21T23:14:53.275066+0000","last_active":"2024-09-21T23:14:53.275066+0000","last_peered":"2024-09-21T23:14:53.275066+0000","last_clean":"2024-09-21T23:14:53.275066+0000","last_became_active":"2024-09-21T23:14:51.503577+0000","last_became_peered":"2024-09-21T23:14:51.503577+0000","last_unstale":"2024-09-21T23:14:53.275066+0000","last_undegraded":"2024-09-21T23:14:53.275066+0000","last_fullsized":"2024-09-21T23:14:53.275066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:34:53.168488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.269144+0000","last_change":"2024-09-21T23:14:53.267881+0000","last_active":"2024-09-21T23:14:53.269144+0000","last_peered":"2024-09-21T23:14:53.269144+0000","last_clean":"2024-09-21T23:14:53.269144+0000","last_became_active":"2024-09-21T23:14:51.255582+0000","last_became_peered":"2024-09-21T23:14:51.255582+0000","last_unstale":"2024-09-21T23:14:53.269144+0000","last_undegraded":"2024-09-21T23:14:53.269144+0000","last_fullsized":"2024-09-21T23:14:53.269144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:54.091377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:54.112260+0000","last_change":"2024-09-21T23:14:54.112260+0000","last_active":"2024-09-21T23:14:54.112260+0000","last_peered":"2024-09-21T23:14:54.112260+0000","last_clean":"2024-09-21T23:14:54.112260+0000","last_became_active":"2024-09-21T23:14:51.257663+0000","last_became_peered":"2024-09-21T23:14:51.257663+0000","last_unstale":"2024-09-21T23:14:54.112260+0000","last_undegraded":"2024-09-21T23:14:54.112260+0000","last_fullsized":"2024-09-21T23:14:54.112260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:20.628422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262129+0000","last_change":"2024-09-21T23:14:53.262129+0000","last_active":"2024-09-21T23:14:53.262129+0000","last_peered":"2024-09-21T23:14:53.262129+0000","last_clean":"2024-09-21T23:14:53.262129+0000","last_became_active":"2024-09-21T23:14:51.249890+0000","last_became_peered":"2024-09-21T23:14:51.249890+0000","last_unstale":"2024-09-21T23:14:53.262129+0000","last_undegraded":"2024-09-21T23:14:53.262129+0000","last_fullsized":"2024-09-21T23:14:53.262129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:51:33.714563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.266479+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:14:53.266479+0000","last_peered":"2024-09-21T23:14:53.266479+0000","last_clean":"2024-09-21T23:14:53.266479+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:14:53.266479+0000","last_undegraded":"2024-09-21T23:14:53.266479+0000","last_fullsized":"2024-09-21T23:14:53.266479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.260975+0000","last_change":"2024-09-21T23:14:46.402802+0000","last_active":"2024-09-21T23:14:53.260975+0000","last_peered":"2024-09-21T23:14:53.260975+0000","last_clean":"2024-09-21T23:14:53.260975+0000","last_became_active":"2024-09-21T23:14:46.400426+0000","last_became_peered":"2024-09-21T23:14:46.400426+0000","last_unstale":"2024-09-21T23:14:53.260975+0000","last_undegraded":"2024-09-21T23:14:53.260975+0000","last_fullsized":"2024-09-21T23:14:53.260975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:34.310611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:14:58.436 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:14:58.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T23:14:58.725 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 20 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:fsid 309803bf-3365-47ac-8bf6-365125beceb1 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-09-21T23:14:32.028556+0000 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:modified 2024-09-21T23:14:56.731646+0000 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:crush_version 5 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:full_ratio 0.95 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:backfillfull_ratio 0.9 2024-09-21T23:14:58.726 INFO:teuthology.orchestra.run.smithi053.stdout:nearfull_ratio 0.85 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:require_min_compat_client luminous 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:min_compat_client jewel 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:require_osd_release squid 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:stretch_mode_enabled false 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:max_osd 6 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/91058490,v1:172.21.15.53:6817/91058490] [v2:172.21.15.53:6818/91058490,v1:172.21.15.53:6819/91058490] exists,up a24df615-4443-4f27-888f-d1e4e7affbb3 2024-09-21T23:14:58.727 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/2712042978,v1:172.21.15.53:6809/2712042978] [v2:172.21.15.53:6810/2712042978,v1:172.21.15.53:6811/2712042978] exists,up d6765c2a-98b2-4a2b-8d8c-f3963575eb64 2024-09-21T23:14:58.728 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/1166517858,v1:172.21.15.53:6801/1166517858] [v2:172.21.15.53:6802/1166517858,v1:172.21.15.53:6803/1166517858] exists,up 9753272a-58a9-4366-87d7-550c49134023 2024-09-21T23:14:58.728 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6809/1039392120,v1:172.21.15.144:6811/1039392120] [v2:172.21.15.144:6814/1039392120,v1:172.21.15.144:6816/1039392120] exists,up 24d987f9-f2cd-4ccd-9dc8-fea64467d92d 2024-09-21T23:14:58.728 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/2764095640,v1:172.21.15.144:6810/2764095640] [v2:172.21.15.144:6812/2764095640,v1:172.21.15.144:6813/2764095640] exists,up c1c30a49-7f26-4264-a1a9-160dc552953d 2024-09-21T23:14:58.728 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/1732973431,v1:172.21.15.144:6801/1732973431] [v2:172.21.15.144:6802/1732973431,v1:172.21.15.144:6803/1732973431] exists,up f24e9341-be4e-4be2-b180-7ebd3f705e9b 2024-09-21T23:14:58.743 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/91058490,v1:172.21.15.53:6817/91058490] [v2:172.21.15.53:6818/91058490,v1:172.21.15.53:6819/91058490] exists,up a24df615-4443-4f27-888f-d1e4e7affbb3', 'osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/2712042978,v1:172.21.15.53:6809/2712042978] [v2:172.21.15.53:6810/2712042978,v1:172.21.15.53:6811/2712042978] exists,up d6765c2a-98b2-4a2b-8d8c-f3963575eb64', 'osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/1166517858,v1:172.21.15.53:6801/1166517858] [v2:172.21.15.53:6802/1166517858,v1:172.21.15.53:6803/1166517858] exists,up 9753272a-58a9-4366-87d7-550c49134023', 'osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6809/1039392120,v1:172.21.15.144:6811/1039392120] [v2:172.21.15.144:6814/1039392120,v1:172.21.15.144:6816/1039392120] exists,up 24d987f9-f2cd-4ccd-9dc8-fea64467d92d', 'osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/2764095640,v1:172.21.15.144:6810/2764095640] [v2:172.21.15.144:6812/2764095640,v1:172.21.15.144:6813/2764095640] exists,up c1c30a49-7f26-4264-a1a9-160dc552953d', 'osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/1732973431,v1:172.21.15.144:6801/1732973431] [v2:172.21.15.144:6802/1732973431,v1:172.21.15.144:6803/1732973431] exists,up f24e9341-be4e-4be2-b180-7ebd3f705e9b'] 2024-09-21T23:14:58.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mon.a", 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster": "ceph", 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "admin_socket_mode": "", 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_client_required": "cephx, none", 2024-09-21T23:14:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_cluster_required": "cephx", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_debug": "true", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_service_required": "cephx", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_supported": "cephx", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio": "true", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T23:14:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_reap_max": "16", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_async_discard": "false", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_block_size": "4096", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio": "false", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_enable_discard": "false", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_flock_retry": "3", 2024-09-21T23:14:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T23:14:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_inject_crash": "0", 2024-09-21T23:14:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T23:14:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring": "false", 2024-09-21T23:14:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring_hipri": "false", 2024-09-21T23:14:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T23:14:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T23:14:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_type": "", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_allocator": "hybrid", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_buffered_io": "true", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T23:14:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T23:14:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_replay_recovery": "false", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_sync_write": "false", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_allocator": "hybrid", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T23:14:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bdev_label_multi": "true", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_create": "true", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_create": "false", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_path": "", 2024-09-21T23:14:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_size": "0", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_path": "", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_size": "107374182400", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_create": "false", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_path": "", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs": "true", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T23:14:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_autotune": "true", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size": "0", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T23:14:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_type": "2q", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_clone_cow": "true", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T23:14:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_mode": "none", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_freelist": "false", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T23:14:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T23:14:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fail_eio": "false", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T23:14:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T23:14:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size": "0", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T23:14:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T23:14:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_mem": "512", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T23:14:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_tracing": "false", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T23:14:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T23:14:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_cluster_require_version": "2", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_require_signatures": "false", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_require_version": "2", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_service_require_signatures": "false", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_service_require_version": "2", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_sign_messages": "true", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "chdir": "", 2024-09-21T23:14:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_acl_type": "", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_asio_thread_count": "2", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_cache_mid": "0.750000", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_cache_size": "16384", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_caps_release_delay": "5", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_check_pool_perm": "true", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_force_sync_read": "false", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_getattr_caps": "false", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_inject_features": "", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T23:14:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_die_on_failed_remount": "false", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_dirsize_rbytes": "true", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_force_lazyio": "false", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_fs": "", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_inject_release_failure": "false", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_max_inline_size": "4096", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mds_namespace": "", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_metadata": "", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_gid": "-1", 2024-09-21T23:14:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_timeout": "300", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_uid": "-1", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mountpoint": "/", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_notify_timeout": "10", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc": "true", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_objects": "1000", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_size": "209715200", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_permissions": "true", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_quota": "true", 2024-09-21T23:14:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_quota_df": "true", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_max_bytes": "0", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_max_periods": "4", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_min": "131072", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_reconnect_stale": "false", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_shutdown_timeout": "30", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_snapdir": ".snap", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_tick_interval": "1", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_trace": "", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_use_faked_inos": "false", 2024-09-21T23:14:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_use_random_mds": "false", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog": "false", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog_port": "12201", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_monitors": "default=true", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog": "false", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog_level": "info", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "-", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_network": "", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_network_interface": "", 2024-09-21T23:14:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_isal": "false", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_level": "5", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zstd_level": "1", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_osd_stat_interval": "0", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_seastar_cpu_cores": "", 2024-09-21T23:14:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_seastar_num_threads": "0", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location": "", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location_hook": "", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location_hook_timeout": "10", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "daemonize": "false", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "daos_pool": "tank", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_alienstore": "0/5", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asok": "1/5", 2024-09-21T23:14:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asok_assert_abort": "false", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_auth": "20/20", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bdev": "1/3", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bluefs": "1/5", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bluestore": "1/5", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_buffer": "0/1", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cephsqlite": "0/5", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_client": "0/5", 2024-09-21T23:14:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_compressor": "1/5", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_context": "0/1", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_crimson_interrupt": "0/5", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_crush": "1/1", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_crypto": "1/5", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cyanstore": "0/5", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_dpdk": "1/5", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_eventtrace": "1/5", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_filer": "0/1", 2024-09-21T23:14:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_filestore": "1/3", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_finisher": "1/1", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_fuse": "1/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_javaclient": "1/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_journal": "1/3", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_journaler": "0/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_kstore": "1/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_lockdep": "0/1", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mclock": "1/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds": "1/5", 2024-09-21T23:14:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_balancer": "1/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_locker": "1/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_log": "1/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_migrator": "1/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_quiesce": "3/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_memstore": "1/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mgr": "2/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mgrc": "1/5", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mon": "20/20", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_monc": "0/10", 2024-09-21T23:14:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_ms": "1/1", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_none": "0/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objclass": "0/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objectcacher": "0/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objecter": "0/1", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_optracker": "0/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_osd": "1/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_paxos": "20/20", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_perfcounter": "1/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_prioritycache": "1/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rados": "0/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd": "0/5", 2024-09-21T23:14:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_replay": "0/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_refs": "0/0", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_reserver": "1/1", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw": "1/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_access": "1/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_flight": "1/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-21T23:14:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_sync": "1/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rocksdb": "4/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_backref": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_cache": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_device": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_epm": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_journal": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_lba": "0/5", 2024-09-21T23:14:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_odata": "0/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_omap": "0/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_onode": "0/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_t": "0/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_tm": "0/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_striper": "0/1", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_test": "0/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_throttle": "1/1", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_timer": "0/1", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_tp": "0/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_trace": "1/5", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T23:14:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "device_failure_prediction_mode": "none", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_graylog": "false", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_journald": "false", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_stderr": "true", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_syslog": "false", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event_tracing": "false", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_http_port": "9926", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_prio_limit": "5", 2024-09-21T23:14:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_sort_metrics": "true", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_stats_period": "5", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "fake_statfs_for_testing": "0", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "fatal_signal_handlers": "true", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "filer_max_purge_ops": "10", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "filer_max_truncate_ops": "128", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_blackhole": "false", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_btrfs_snap": "true", 2024-09-21T23:14:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_caller_concurrency": "10", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_omap_check": "false", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_verify_split": "false", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_dump_file": "", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T23:14:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fadvise": "true", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fail_eio": "true", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fd_cache_size": "128", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fiemap": "false", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_inject_stall": "0", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_parallel": "false", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_trailing": "false", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_writeahead": "false", 2024-09-21T23:14:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_kill_at": "0", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T23:14:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_merge_threshold": "-10", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_odsync_write": "false", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_backend_path": "", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T23:14:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_threads": "2", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_punch_hole": "false", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T23:14:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_ops": "50", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_seek_data_hole": "false", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_sloppy_crc": "false", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_splice": "false", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_split_multiple": "2", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_split_rand_factor": "20", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_update_to": "1000", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T23:14:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_xfs_extsize": "false", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_zfs_snap": "false", 2024-09-21T23:14:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "fio_dir": "/tmp/fio", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "309803bf-3365-47ac-8bf6-365125beceb1", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_allow_other": "true", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_big_writes": "true", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_debug": "false", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_default_permissions": "false", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_disable_pagecache": "false", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_max_write": "0", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_multithreaded": "true", 2024-09-21T23:14:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_require_active_mds": "true", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_set_user_groups": "true", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_move": "true", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_read": "true", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_write": "true", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "gss_target_name": "ceph", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_file": "", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_inject_failure": "0", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_interval": "5", 2024-09-21T23:14:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "host": "smithi053", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T23:14:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "inject_early_sigterm": "false", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "jaeger_agent_port": "6799", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "jaeger_tracing_enable": "false", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_aio": "true", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_align_min_size": "65536", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_block_align": "true", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_block_size": "4096", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_dio": "true", 2024-09-21T23:14:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_discard": "false", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_force_aio": "false", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_ignore_corruption": "false", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_max_write_entries": "100", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_replay_from": "0", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T23:14:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_write_header_frequency": "0", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_zero_on_create": "false", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_prefetch_periods": "10", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_prezero_periods": "5", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_write_head_interval": "15", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "keyfile": "", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_backend": "rocksdb", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T23:14:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_max_bytes": "67108864", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_max_ops": "512", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_onode_map_size": "1024", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_sync_transaction": "false", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "librados_thread_count": "2", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "lockdep": "false", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "lockdep_force_backtrace": "false", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "log_coarse_timestamps": "true", 2024-09-21T23:14:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_flush_on_exit": "false", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_graylog_port": "12201", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_max_new": "1000", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_max_recent": "10000", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_stderr_prefix": "", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": "true", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_graylog": "false", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_journald": "false", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_stderr": "false", 2024-09-21T23:14:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_syslog": "false", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_action_on_write_error": "1", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_alternate_name_max": "8192", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_asio_thread_count": "2", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_export_pin": "true", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T23:14:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_interval": "10", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_max": "-1", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_max_until": "-1", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_merge_size": "50", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T23:14:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_mode": "0", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_overload_epochs": "2", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_bits": "3", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_size": "10000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T23:14:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_mid": "0.700000", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_quiesce_delay": "0", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T23:14:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_interval": "1", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T23:14:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_auth_pins": "false", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_frag": "false", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_scatterstat": "false", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_subtrees": "false", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_default_dir_hash": "2", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_defer_session_stale": "true", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_max_entries": "0", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_prefetch": "true", 2024-09-21T23:14:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_early_reply": "true", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_enable_op_tracker": "true", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_enforce_unique_name": "true", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T23:14:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_health_dummy": "false", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T23:14:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_join_fs": "", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_journal_format": "1", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_create_at": "0", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_export_at": "0", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_import_at": "0", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_at": "0", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T23:14:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_link_at": "0", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_openc_at": "0", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_rename_at": "0", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_shutdown_at": "0", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_event_large_threshold": "524288", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_max_events": "-1", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_max_segments": "128", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_pause": "false", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_segment_size": "0", 2024-09-21T23:14:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_trim_threshold": "128", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_completed_requests": "100000", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_export_size": "20971520", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_file_recover": "32", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_files": "64", 2024-09-21T23:14:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_ops": "8192", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_metrics_update_interval": "2", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_min_caps_per_client": "100", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_numa_node": "-1", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_duration": "600", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_size": "20", 2024-09-21T23:14:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_log_threshold": "5", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_ping_grace": "15", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_ping_interval": "5", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_caps": "30000", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T23:14:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_replay_interval": "1.000000", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_root_ino_gid": "0", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_root_ino_uid": "0", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T23:14:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_shutdown_check": "0", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_skip_ino": "0", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_min_uid": "0", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_rstat": "false", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_symlink_recovery": "true", 2024-09-21T23:14:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_thrash_exports": "0", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_thrash_fragments": "0", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_tick_interval": "5.000000", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_valgrind_exit": "false", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_verify_backtrace": "1", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_verify_scatter": "false", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_wipe_sessions": "false", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool_debug": "false", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T23:14:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_page_set": "false", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_page_size": "65536", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_bytes": "134217728", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_messages": "512", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_disabled_modules": "", 2024-09-21T23:14:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_enable_op_tracker": "true", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mds_messages": "128", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mon_messages": "128", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_op_history_duration": "600", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_op_history_size": "20", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-21T23:14:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_op_log_threshold": "5", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_osd_messages": "8192", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_pool": "true", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_standby_modules": "true", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_stats_period": "5", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_stats_threshold": "5", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_tick_period": "2", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T23:14:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_allow_pool_delete": "true", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_bytes": "104857600", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T23:14:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_target_rank": "-1", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T23:14:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T23:14:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T23:14:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T23:14:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T23:14:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T23:14:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T23:14:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T23:14:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_start": "false", 2024-09-21T23:14:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_trim": "true", 2024-09-21T23:14:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T23:14:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T23:14:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T23:14:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cpu_threads": "4", 2024-09-21T23:14:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_avail_crit": "5", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_avail_warn": "5", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_json": "false", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T23:14:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_extra_checks": "false", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_squid": "false", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_down_uptime_grace": "60", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_election_default_strategy": "1", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_election_timeout": "5.000000", 2024-09-21T23:14:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_enable_op_tracker": "true", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_fake_pool_delete": "false", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_force_quorum_join": "false", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_log_update_period": "5", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_max_detail": "50", 2024-09-21T23:14:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog": "true", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_host": "172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.144", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_host_override": "", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_initial_members": "", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T23:14:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease": "5.000000", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_log_full_interval": "50", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_log_max": "10000", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_log_max_summary": "50", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_log_epochs": "500", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T23:14:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_osd": "10000", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_memory_autotune": "true", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_memory_target": "2147483648", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_digest_period": "5", 2024-09-21T23:14:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_complaint_time": "30", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_duration": "600", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_size": "20", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_log_threshold": "5", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T23:14:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_cache_size": "500", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T23:14:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T23:14:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_report_timeout": "900", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T23:14:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_max_osds": "4", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T23:14:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_interval": "86400", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_max_keys": "100", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_timeout": "300", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_session_timeout": "300", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_smart_report_timeout": "5", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_pool_size": "4", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T23:14:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_debug": "false", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_tick_interval": "5", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T23:14:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_older_version": "true", 2024-09-21T23:14:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "monmap": "", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T23:14:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_tracing_enabled": "false", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_op_threads": "3", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_cm": "false", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_device_name": "", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T23:14:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_sl": "3", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_type": "ib", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_reap_threshold": "5", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_before_connect": "false", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_ipv4": "true", 2024-09-21T23:14:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_ipv6": "false", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_msgr1": "true", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_msgr2": "true", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_port_max": "7568", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_port_min": "6800", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_retry_count": "3", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_retry_delay": "5", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_client": "false", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mds": "false", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mgr": "false", 2024-09-21T23:14:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mon": "false", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_osd": "false", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_client_mode": "crc secure", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_cluster_type": "", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_compress_secure": "false", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_crc_data": "true", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_crc_header": "true", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_bug": "true", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_old_message": "true", 2024-09-21T23:14:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T23:14:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hugepages": "", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_lro": "true", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_pmd": "", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_port_id": "0", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dump_on_send": "false", 2024-09-21T23:14:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_type": "", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_network_congestion": "0", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_max_accept_failures": "4", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_max_backoff": "15.000000", 2024-09-21T23:14:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_osd_compress_mode": "none", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_pq_min_cost": "65536", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_public_type": "", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_service_mode": "crc secure", 2024-09-21T23:14:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_nodelay": "true", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_type": "async+posix", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "no_config_file": "false", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inflight_ops": "1024", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T23:14:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_timeout": "10.000000", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "objectstore_blackhole": "false", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "openssl_engine_opts": "", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_max_ops": "4", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T23:14:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_slop": "0.020000", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_asio_thread_count": "2", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_scan_max": "512", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_scan_min": "64", 2024-09-21T23:14:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T23:14:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_peering": "false", 2024-09-21T23:14:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T23:14:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_beacon_report_interval": "300", 2024-09-21T23:14:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_duration": "30", 2024-09-21T23:14:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T23:14:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T23:14:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_blkin_trace_all": "false", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T23:14:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_update_on_start": "true", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_message_cap": "256", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_op_priority": "63", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_watch_timeout": "30", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_command_max_records": "256", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_compact_on_start": "false", 2024-09-21T23:14:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_update_on_start": "true", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T23:14:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_op_order": "false", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T23:14:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_shutdown": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_trim_objects": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T23:14:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_default_notify_timeout": "30", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T23:14:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_enable_op_tracker": "true", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_info": "true", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown": "true", 2024-09-21T23:14:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_function_tracing": "false", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_grace": "20", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_interval": "6", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T23:14:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_stale": "600", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T23:14:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal_size": "5120", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_kill_backfill_at": "0", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_cache_size": "50", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_dedup": "true", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_message_max": "40", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_attr_name_len": "100", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_attr_size": "0", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_backfills": "1", 2024-09-21T23:14:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_markdown_count": "5", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_markdown_period": "600", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_name_len": "2048", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_size": "134217728", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pgls": "1024", 2024-09-21T23:14:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_push_cost": "8388608", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_push_objects": "10", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_scrubs": "3", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_write_size": "90", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T23:14:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_profile": "balanced", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T23:14:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_base": "805306368", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target": "4294967296", 2024-09-21T23:14:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target_autotune": "false", 2024-09-21T23:14:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T23:14:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T23:14:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_min_recovery_priority": "0", 2024-09-21T23:14:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T23:14:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T23:14:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_report_interval": "5", 2024-09-21T23:14:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T23:14:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T23:14:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_num_cache_shards": "32", 2024-09-21T23:14:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T23:14:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_node": "-1", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objecter_finishers": "1", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore": "bluestore", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore_fuse": "false", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore_tracing": "false", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_duration": "600", 2024-09-21T23:14:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_size": "20", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_log_threshold": "5", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards": "0", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T23:14:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T23:14:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T23:14:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T23:14:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_thread_timeout": "15", 2024-09-21T23:14:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_open_classes_on_start": "true", 2024-09-21T23:14:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_os_flags": "0", 2024-09-21T23:14:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_peering_op_priority": "255", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_delete_priority": "5", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T23:14:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_crimson": "false", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T23:14:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flags": "0", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_min_size": "0", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T23:14:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T23:14:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_read_ratio": "70", 2024-09-21T23:14:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_size": "2", 2024-09-21T23:14:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_type": "replicated", 2024-09-21T23:14:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T23:14:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T23:14:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_cost": "20971520", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active": "0", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T23:14:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_op_priority": "3", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_priority": "5", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T23:14:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_repair_during_recovery": "false", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T23:14:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_cost": "52428800", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_end_hour": "0", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T23:14:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_priority": "5", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_skip_data_digest": "false", 2024-09-21T23:14:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_smart_report_timeout": "5", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_priority": "5", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_target_transaction_size": "30", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T23:14:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tracing": "false", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_use_stale_snap": "false", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T23:14:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_kill_at": "0", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_max_join_drift": "10", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_min": "500", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_min_wait": "0.050000", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_max": "500", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_min": "250", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_stash_full_interval": "25", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_trim_max": "500", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_trim_min": "250", 2024-09-21T23:14:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "perf": "true", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "-", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrv": "", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "public_bind_addr": "-", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "public_network": "", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "public_network_interface": "", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "qat_compressor_busy_polling": "false", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "qat_compressor_enabled": "false", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T23:14:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_mon_op_timeout": "0", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_osd_op_timeout": "0", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_tracing": "false", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_atime_update_interval": "60", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache": "true", 2024-09-21T23:14:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_policy": "writearound", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_size": "33554432", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_compression_hint": "none", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_clone_format": "auto", 2024-09-21T23:14:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_data_pool": "", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_features": "61", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_format": "2", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_map_options": "", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_order": "22", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_pool": "rbd", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_stripe_count": "0", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T23:14:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_io_scheduler": "simple", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T23:14:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_order": "24", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_pool": "", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_splay_width": "4", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T23:14:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T23:14:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T23:14:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_op_threads": "1", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_plugins": "", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T23:14:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T23:14:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T23:14:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_tracing": "false", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_validate_names": "true", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_validate_pool": "true", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "restapi_base_url": "", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "restapi_log_level": "", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T23:14:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_admin_entry": "admin", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_backend_store": "rados", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_barbican_url": "", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_beast_enable_async": "true", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_counters_cache": "false", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T23:14:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_enabled": "true", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_lru_size": "25000", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_config_store": "rados", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_content_length_compat": "false", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cross_domain_policy": "", 2024-09-21T23:14:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T23:14:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T23:14:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T23:14:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T23:14:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d4n_port": "6379", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_window": "30", 2024-09-21T23:14:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T23:14:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T23:14:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dns_name": "", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dns_s3website_name": "", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_ops_log": "false", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T23:14:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_static_website": "false", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_usage_log": "false", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_expose_bucket": "false", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_extended_http_attrs": "", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_filter": "none", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T23:14:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T23:14:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T23:14:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T23:14:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T23:14:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_objs": "32", 2024-09-21T23:14:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T23:14:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T23:14:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T23:14:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T23:14:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T23:14:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T23:14:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T23:14:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_init_timeout": "300", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_kafka_connection_idle": "300", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T23:14:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_password": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_project": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_token": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_user": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_api_version": "2", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T23:14:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_url": "", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_objs": "32", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T23:14:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_worker": "3", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_uri": "", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T23:14:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_http_headers": "", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attr_size": "0", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T23:14:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_listing_results": "1000", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_notify_retries": "10", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T23:14:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T23:14:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T23:14:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T23:14:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T23:14:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T23:14:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T23:14:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T23:14:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_num_control_oids": "8", 2024-09-21T23:14:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_numa_node": "-1", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_tracing": "false", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_token": "", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_url": "", 2024-09-21T23:14:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_rados": "false", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T23:14:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_posix_cache_lanes": "3", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_posix_cache_partitions": "3", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_print_continue": "true", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T23:14:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_tracing": "false", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm": "", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm_id": "", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_region": "", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T23:14:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_request_uri": "", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_resolve_cname": "false", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_run_sync_thread": "true", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-21T23:14:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_script_uri": "", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_service_provider_name": "", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T23:14:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_client_id": "", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_client_secret": "", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_entry": "sts", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_key": "sts", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_auth_url": "", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_custom_header": "", 2024-09-21T23:14:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_need_stats": "true", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_tenant_name": "", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_url": "", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_lease_period": "120", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T23:14:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_thread_pool_size": "512", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_comment": "", 2024-09-21T23:14:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_createby": "", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_encoding": "", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_flag": "false", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_origin": "", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_tracker": "", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_max_shards": "32", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_use_opa_authz": "false", 2024-09-21T23:14:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_counters_cache": "false", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T23:14:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_unique_email": "true", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_verify_ssl": "true", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone": "", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone_id": "", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup": "", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup_id": "", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T23:14:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_block_size": "4096", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T23:14:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_partition_filters": "false", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_perf": "false", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "run_dir": "/var/run/ceph", 2024-09-21T23:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_block_create": "true", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_device_size": "53687091200", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_full_integrity_check": "false", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T23:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_main_device_type": "SSD", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_segment_size": "67108864", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "set_keepcaps": "false", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "setgroup": "", 2024-09-21T23:14:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "setuser": "", 2024-09-21T23:14:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "setuser_match_path": "", 2024-09-21T23:14:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T23:14:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "thp": "false", 2024-09-21T23:14:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "threadpool_default_timeout": "60", 2024-09-21T23:14:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T23:14:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "throttler_perf_counter": "true" 2024-09-21T23:14:59.044 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:14:59.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T23:14:59.191 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a: {} 2024-09-21T23:14:59.192 INFO:teuthology.orchestra.run.smithi053.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:14:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b: {} 2024-09-21T23:14:59.204 INFO:teuthology.orchestra.run.smithi053.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:14:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c: {} 2024-09-21T23:14:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:14:59.230 DEBUG:teuthology.orchestra.run.smithi053:> type ceph-objectstore-tool 2024-09-21T23:14:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:14:59.234 DEBUG:teuthology.orchestra.run.smithi144:> type ceph-objectstore-tool 2024-09-21T23:14:59.240 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:14:59.240 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-21T23:14:59.253 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-21T23:14:59.253 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T23:14:59.253 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:14:59.254 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:14:59.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:14:59.254 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T23:14:59.254 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T23:14:59.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:14:59.255 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T23:14:59.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:14:59.256 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T23:14:59.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:14:59.256 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-21T23:14:59.278 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:14:59.276+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:14:59.310 INFO:teuthology.orchestra.run.smithi170.stdout:mapped writes DISABLED 2024-09-21T23:14:59.311 INFO:teuthology.orchestra.run.smithi170.stdout:mapped reads DISABLED 2024-09-21T23:14:59.311 INFO:teuthology.orchestra.run.smithi170.stdout:Seed set to 3567 2024-09-21T23:14:59.311 INFO:teuthology.orchestra.run.smithi170.stdout:rbd-nbd mode enabled 2024-09-21T23:14:59.356 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:14:59.351+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:14:59.402 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:14:59.402 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:14:59.402 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:14:59.402 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:14:59.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:14:59.419 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:14:59.419 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:14:59.439 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:14:59.440 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:14:59.457 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:14:59.451+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:14:59.472 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:14:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:14:59.508 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:14:59.508 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:14:59.537 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:14:59.537 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:14:59.543 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:14:59.543 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:14:59.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:14:59.558 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:14:59.551+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:14:59.579 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:14:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:14:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:14:59.587 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:14:59.608 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-21T23:14:58.842985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":176900,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566054140,"statfs":{"total":579820584960,"available":579639439360,"internally_reserved":0,"allocated":2277376,"data_stored":1539058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.591665"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262843+0000","last_change":"2024-09-21T23:14:53.262843+0000","last_active":"2024-09-21T23:14:53.262843+0000","last_peered":"2024-09-21T23:14:53.262843+0000","last_clean":"2024-09-21T23:14:53.262843+0000","last_became_active":"2024-09-21T23:14:51.360795+0000","last_became_peered":"2024-09-21T23:14:51.360795+0000","last_unstale":"2024-09-21T23:14:53.262843+0000","last_undegraded":"2024-09-21T23:14:53.262843+0000","last_fullsized":"2024-09-21T23:14:53.262843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:10.419450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016301600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.274401+0000","last_change":"2024-09-21T23:14:53.274401+0000","last_active":"2024-09-21T23:14:53.274401+0000","last_peered":"2024-09-21T23:14:53.274401+0000","last_clean":"2024-09-21T23:14:53.274401+0000","last_became_active":"2024-09-21T23:14:51.253820+0000","last_became_peered":"2024-09-21T23:14:51.253820+0000","last_unstale":"2024-09-21T23:14:53.274401+0000","last_undegraded":"2024-09-21T23:14:53.274401+0000","last_fullsized":"2024-09-21T23:14:53.274401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:51:57.046270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262402+0000","last_change":"2024-09-21T23:14:53.262402+0000","last_active":"2024-09-21T23:14:53.262402+0000","last_peered":"2024-09-21T23:14:53.262402+0000","last_clean":"2024-09-21T23:14:53.262402+0000","last_became_active":"2024-09-21T23:14:51.500246+0000","last_became_peered":"2024-09-21T23:14:51.500246+0000","last_unstale":"2024-09-21T23:14:53.262402+0000","last_undegraded":"2024-09-21T23:14:53.262402+0000","last_fullsized":"2024-09-21T23:14:53.262402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:33:42.165228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012328759999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.275066+0000","last_change":"2024-09-21T23:14:53.275066+0000","last_active":"2024-09-21T23:14:53.275066+0000","last_peered":"2024-09-21T23:14:53.275066+0000","last_clean":"2024-09-21T23:14:53.275066+0000","last_became_active":"2024-09-21T23:14:51.503577+0000","last_became_peered":"2024-09-21T23:14:51.503577+0000","last_unstale":"2024-09-21T23:14:53.275066+0000","last_undegraded":"2024-09-21T23:14:53.275066+0000","last_fullsized":"2024-09-21T23:14:53.275066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:34:53.168488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.269144+0000","last_change":"2024-09-21T23:14:53.267881+0000","last_active":"2024-09-21T23:14:53.269144+0000","last_peered":"2024-09-21T23:14:53.269144+0000","last_clean":"2024-09-21T23:14:53.269144+0000","last_became_active":"2024-09-21T23:14:51.255582+0000","last_became_peered":"2024-09-21T23:14:51.255582+0000","last_unstale":"2024-09-21T23:14:53.269144+0000","last_undegraded":"2024-09-21T23:14:53.269144+0000","last_fullsized":"2024-09-21T23:14:53.269144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:16:54.091377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:54.112260+0000","last_change":"2024-09-21T23:14:54.112260+0000","last_active":"2024-09-21T23:14:54.112260+0000","last_peered":"2024-09-21T23:14:54.112260+0000","last_clean":"2024-09-21T23:14:54.112260+0000","last_became_active":"2024-09-21T23:14:51.257663+0000","last_became_peered":"2024-09-21T23:14:51.257663+0000","last_unstale":"2024-09-21T23:14:54.112260+0000","last_undegraded":"2024-09-21T23:14:54.112260+0000","last_fullsized":"2024-09-21T23:14:54.112260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:20.628422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.262129+0000","last_change":"2024-09-21T23:14:53.262129+0000","last_active":"2024-09-21T23:14:53.262129+0000","last_peered":"2024-09-21T23:14:53.262129+0000","last_clean":"2024-09-21T23:14:53.262129+0000","last_became_active":"2024-09-21T23:14:51.249890+0000","last_became_peered":"2024-09-21T23:14:51.249890+0000","last_unstale":"2024-09-21T23:14:53.262129+0000","last_undegraded":"2024-09-21T23:14:53.262129+0000","last_fullsized":"2024-09-21T23:14:53.262129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:51:33.714563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.266479+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:14:53.266479+0000","last_peered":"2024-09-21T23:14:53.266479+0000","last_clean":"2024-09-21T23:14:53.266479+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:14:53.266479+0000","last_undegraded":"2024-09-21T23:14:53.266479+0000","last_fullsized":"2024-09-21T23:14:53.266479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T23:14:53.260975+0000","last_change":"2024-09-21T23:14:46.402802+0000","last_active":"2024-09-21T23:14:53.260975+0000","last_peered":"2024-09-21T23:14:53.260975+0000","last_clean":"2024-09-21T23:14:53.260975+0000","last_became_active":"2024-09-21T23:14:46.400426+0000","last_became_peered":"2024-09-21T23:14:46.400426+0000","last_unstale":"2024-09-21T23:14:53.260975+0000","last_undegraded":"2024-09-21T23:14:53.260975+0000","last_fullsized":"2024-09-21T23:14:53.260975+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:34.310611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":774144,"data_stored":650082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:14:59.659 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:14:59.651+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:14:59.690 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:14:59.690 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:14:59.703 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:14:59.759 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:14:59.755+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:14:59.861 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:14:59.860+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:14:59.865 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:14:59.865 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:14:59.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:14:59.965 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:14:59.964+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:00.041 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:00.041 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:00.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:00.063 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:00.059+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:00.164 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:00.159+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:00.221 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:00.221 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:00.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:00.265 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:00.259+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:00.296 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:15:00.365 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:00.364+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:00.373 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:00.373 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:00.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:00.466 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:00.459+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:00.545 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:00.545 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:00.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:00.568 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:00.563+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:00.669 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:00.668+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:00.679 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:00.679 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:00.694 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:00.769 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:00.763+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:00.825 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:00.825 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:00.840 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:00.870 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:00.868+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:00.970 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:00.963+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:01.005 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:01.005 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:01.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:01.111 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:01.107+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:01.172 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:01.167+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:01.188 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:01.188 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:01.202 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:01.291 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:01.287+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:01.374 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:01.367+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:01.379 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:01.380 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:01.393 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:01.479 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:01.475+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:01.561 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:01.561 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:01.576 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:01.576 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:01.572+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:01.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:01.676 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:01.671+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:01.740 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:01.741 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:01.755 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:01.763 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:15:01.763 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:15:01.779 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:01.780 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:01.798 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:01.799 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:01.795+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:01.820 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:01.820 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:01.838 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:01.838 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:01.857 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:01.877 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:01.871+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:01.924 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:01.924 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:01.924 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:01.924 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:01.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:01.983 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:01.979+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:02.079 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:02.075+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:02.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:02.106 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:02.106 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:02.120 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:02.211 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:02.207+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:02.281 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:02.280+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:02.285 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:02.285 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:02.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:02.285 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:02.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.285 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4203.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T23:15:02.285 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:51.250220+0000", 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.032546706, 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091289869999999999, 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4203", 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.53:0/1772892663", 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250220+0000", 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250220+0000", 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250229+0000", 2024-09-21T23:15:02.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4800000000000001e-06 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250242+0000", 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3468999999999999e-05 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250270+0000", 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-05 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250295+0000", 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5511999999999999e-05 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250476+0000", 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018087299999999999 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.250487+0000", 2024-09-21T23:15:02.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1076e-05 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.252656+0000", 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0021690110000000002 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.252679+0000", 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3175000000000001e-05 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.257017+0000", 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043374049999999999 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.257347+0000", 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000330568 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.257577+0000", 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023026200000000001 2024-09-21T23:15:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.258947+0000", 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0013697519999999999 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259191+0000", 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024398099999999999 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:02.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259279+0000", 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7378000000000003e-05 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259349+0000", 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0845000000000004e-05 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.293 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop_reply(client.4203.0:1 2.3 e17/16 ondisk, result = 0)", 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:51.259032+0000", 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.023735518000001, 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000330145, 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259032+0000", 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259032+0000", 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259033+0000", 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7630000000000001e-06 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259043+0000", 2024-09-21T23:15:02.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3340000000000008e-06 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259067+0000", 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4324999999999998e-05 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259081+0000", 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3794e-05 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259169+0000", 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8443999999999995e-05 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259183+0000", 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3336000000000001e-05 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.259362+0000", 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179149 2024-09-21T23:15:02.297 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4203.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:51.263079+0000", 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.019687984000001, 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042783699999999998, 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4203", 2024-09-21T23:15:02.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.53:0/1772892663", 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263079+0000", 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263079+0000", 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263081+0000", 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.244e-06 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263094+0000", 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.237e-05 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263125+0000", 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1262e-05 2024-09-21T23:15:02.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263150+0000", 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4589000000000001e-05 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263300+0000", 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015021699999999999 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.263507+0000", 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020715499999999999 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4203.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:51.264103+0000", 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.018663835, 2024-09-21T23:15:02.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041836, 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4203", 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.53:0/1772892663", 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264103+0000", 2024-09-21T23:15:02.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264103+0000", 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264105+0000", 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0700000000000001e-06 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264117+0000", 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1202999999999999e-05 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264147+0000", 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0568999999999998e-05 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264162+0000", 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4732999999999999e-05 2024-09-21T23:15:02.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264329+0000", 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016708200000000001 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:51.264522+0000", 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019270299999999999 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4203.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:52.255122+0000", 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.027645388, 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0064354599999999996, 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4203", 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.53:0/1772892663", 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.307 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255122+0000", 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255122+0000", 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255125+0000", 2024-09-21T23:15:02.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7760000000000002e-06 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255156+0000", 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1952999999999997e-05 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255190+0000", 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3955999999999998e-05 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255212+0000", 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1619000000000001e-05 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255349+0000", 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013743300000000001 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.255360+0000", 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.043e-05 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.258417+0000", 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003056886 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.258604+0000", 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018723200000000001 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.258805+0000", 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020075799999999999 2024-09-21T23:15:02.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.260225+0000", 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001420115 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261371+0000", 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001145952 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261448+0000", 2024-09-21T23:15:02.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6761999999999995e-05 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261557+0000", 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010958799999999999 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop_reply(client.4203.0:5 2.2 e17/16 ondisk, result = 0)", 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:52.261182+0000", 2024-09-21T23:15:02.313 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.021585323, 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038906100000000001, 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261182+0000", 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261182+0000", 2024-09-21T23:15:02.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261184+0000", 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9139999999999998e-06 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261194+0000", 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0253e-05 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261223+0000", 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9442e-05 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261242+0000", 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8453000000000001e-05 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261350+0000", 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010839400000000001 2024-09-21T23:15:02.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261363+0000", 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2396e-05 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:52.261571+0000", 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208209 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4203.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:53.262389+0000", 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.0203778299999993, 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0068145130000000003, 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4203", 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.53:0/1772892663", 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 7 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.318 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262389+0000", 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262389+0000", 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262393+0000", 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0430000000000001e-06 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262412+0000", 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8972999999999999e-05 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262452+0000", 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9582000000000002e-05 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262479+0000", 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6631999999999998e-05 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262668+0000", 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018946600000000001 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.262686+0000", 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7958999999999999e-05 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.265957+0000", 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0032711939999999998 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.266140+0000", 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000182912 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.266362+0000", 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022219399999999999 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.268012+0000", 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00164963 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.269058+0000", 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001045715 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.269142+0000", 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4790999999999995e-05 2024-09-21T23:15:02.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.269204+0000", 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1421999999999996e-05 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop_reply(client.4203.0:7 2.2 e18/16 ondisk, result = 0)", 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:53.268888+0000", 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.0138791559999998, 2024-09-21T23:15:02.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032670900000000001, 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.268888+0000", 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.268888+0000", 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:02.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.268890+0000", 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.796e-06 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.268905+0000", 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5073e-05 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.268929+0000", 2024-09-21T23:15:02.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4207e-05 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.268951+0000", 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1826000000000001e-05 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.269037+0000", 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6328000000000004e-05 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.269049+0000", 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2206e-05 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:53.269215+0000", 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016527299999999999 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:02.328 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:02.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:15:02.330 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:15:02.382 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:02.375+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:02.460 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:02.460 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:02.460 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:02.460 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:02.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:15:02.482 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:02.475+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:02.583 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:02.580+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:02.592 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:02.593 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:02.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:15:02.693 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:02.692+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:02.769 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:02.769 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:02.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:34 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'20, mlcod=14'17)", 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:46.464703+0000", 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.303195898999999, 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024265710000000002, 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464703+0000", 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464703+0000", 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464704+0000", 2024-09-21T23:15:02.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5030000000000001e-06 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464737+0000", 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2805999999999997e-05 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464767+0000", 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9476999999999999e-05 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464780+0000", 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.377e-05 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464928+0000", 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000147999 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.464965+0000", 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6121999999999997e-05 2024-09-21T23:15:02.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.467057+0000", 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020927739999999999 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.467129+0000", 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2119999999999994e-05 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:54 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'32, mlcod=14'29)", 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:46.502707+0000", 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.265191395999999, 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025933470000000002, 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502707+0000", 2024-09-21T23:15:02.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502707+0000", 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502709+0000", 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.72e-06 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502724+0000", 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5237e-05 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502755+0000", 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1112999999999998e-05 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502769+0000", 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3682999999999999e-05 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502946+0000", 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017650200000000001 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.502985+0000", 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9606e-05 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.505204+0000", 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022189369999999998 2024-09-21T23:15:02.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.505301+0000", 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6549e-05 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:55 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'33, mlcod=14'29)", 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:46.504083+0000", 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.263815322999999, 2024-09-21T23:15:02.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026358060000000001, 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504083+0000", 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504083+0000", 2024-09-21T23:15:02.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504085+0000", 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5230000000000001e-06 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504146+0000", 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000004e-05 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504177+0000", 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1146000000000003e-05 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504191+0000", 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3247e-05 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504385+0000", 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019453100000000001 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.504425+0000", 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9480000000000001e-05 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.506621+0000", 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021967229999999998 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.506719+0000", 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7866000000000001e-05 2024-09-21T23:15:02.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:57 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'35, mlcod=14'33)", 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:46.509610+0000", 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.258288599, 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024946080000000002, 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509610+0000", 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509610+0000", 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509612+0000", 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.643e-06 2024-09-21T23:15:02.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509726+0000", 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011388299999999999 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509756+0000", 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0719999999999997e-05 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509769+0000", 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.279e-05 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509954+0000", 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000184475 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.509993+0000", 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9133e-05 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.512003+0000", 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020104480000000002 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.512105+0000", 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101516 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:69 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'43, mlcod=14'39)", 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:46.531675+0000", 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.236224216, 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002551794, 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531675+0000", 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531675+0000", 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531676+0000", 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7230000000000001e-06 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531733+0000", 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6560000000000001e-05 2024-09-21T23:15:02.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531763+0000", 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9683e-05 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531779+0000", 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6586999999999999e-05 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531906+0000", 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012722499999999999 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.531941+0000", 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4576999999999998e-05 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.534159+0000", 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022184090000000002 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.534226+0000", 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.703e-05 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:72 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'46, mlcod=14'43)", 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:46.538372+0000", 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.229526708000002, 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025646470000000002, 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538372+0000", 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538372+0000", 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538374+0000", 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7e-06 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538431+0000", 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6814000000000002e-05 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538454+0000", 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3906000000000001e-05 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538469+0000", 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4133e-05 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538664+0000", 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195155 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.538729+0000", 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5220000000000002e-05 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.540830+0000", 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021014129999999999 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:46.540937+0000", 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010630599999999999 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:129 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'87, mlcod=14'85)", 2024-09-21T23:15:02.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.126924+0000", 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.640975259999999, 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098320049999999996, 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.126924+0000", 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.126924+0000", 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.126925+0000", 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.807e-06 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.126938+0000", 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2952999999999999e-05 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.126968+0000", 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000001e-05 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.126981+0000", 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3737000000000001e-05 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127117+0000", 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013604200000000001 2024-09-21T23:15:02.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127187+0000", 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9906000000000005e-05 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136603+0000", 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0094156880000000002 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136756+0000", 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015242200000000001 2024-09-21T23:15:02.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:130 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'88, mlcod=14'85)", 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.127568+0000", 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.640331115, 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011274816999999999, 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127568+0000", 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127568+0000", 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127569+0000", 2024-09-21T23:15:02.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.747e-06 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127582+0000", 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2713e-05 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127612+0000", 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0046999999999999e-05 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127626+0000", 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3677e-05 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.127988+0000", 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036258999999999998 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128026+0000", 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7669999999999997e-05 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138728+0000", 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010701830000000001 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138842+0000", 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114543 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.806 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:131 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'89, mlcod=14'85)", 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.128254+0000", 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.639644383, 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010725716999999999, 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128254+0000", 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128254+0000", 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128256+0000", 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8899999999999999e-06 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128312+0000", 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5779999999999998e-05 2024-09-21T23:15:02.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128341+0000", 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9258999999999999e-05 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128355+0000", 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.341e-05 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128692+0000", 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033713500000000002 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.128748+0000", 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6178999999999999e-05 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138851+0000", 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010103152000000001 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138980+0000", 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012891199999999999 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.811 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:132 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'90, mlcod=14'85)", 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.129536+0000", 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.638362976, 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.009547982, 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.129536+0000", 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.129536+0000", 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.129538+0000", 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9999999999999999e-06 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.129553+0000", 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5616000000000001e-05 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.129618+0000", 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4324000000000003e-05 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.129635+0000", 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6796e-05 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130195+0000", 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000560685 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130253+0000", 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7747000000000001e-05 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138992+0000", 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0087386019999999998 2024-09-21T23:15:02.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139084+0000", 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2212000000000004e-05 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:133 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'91, mlcod=14'85)", 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.130531+0000", 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.637367403000001, 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0086382219999999992, 2024-09-21T23:15:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130531+0000", 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130531+0000", 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130533+0000", 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7430000000000001e-06 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130545+0000", 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2327000000000001e-05 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130571+0000", 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5069e-05 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.130586+0000", 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5777000000000002e-05 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131156+0000", 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056980900000000001 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131209+0000", 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3389e-05 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139090+0000", 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078809290000000001 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139170+0000", 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9178999999999995e-05 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:134 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'92, mlcod=14'85)", 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.131113+0000", 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.636786052, 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0081447910000000002, 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131113+0000", 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131113+0000", 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131118+0000", 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131175+0000", 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7179999999999998e-05 2024-09-21T23:15:02.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131196+0000", 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0832999999999999e-05 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.131207+0000", 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0857e-05 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132068+0000", 2024-09-21T23:15:02.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086131000000000001 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132121+0000", 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2392999999999998e-05 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139176+0000", 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070559619999999998 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139258+0000", 2024-09-21T23:15:02.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1006000000000003e-05 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:135 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'93, mlcod=14'85)", 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.132278+0000", 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.635620821, 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070613120000000001, 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132278+0000", 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132278+0000", 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132280+0000", 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7570000000000001e-06 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132295+0000", 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5610000000000001e-05 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132320+0000", 2024-09-21T23:15:02.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4686999999999999e-05 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.132333+0000", 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3010000000000001e-05 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133444+0000", 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001111061 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133497+0000", 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3322999999999998e-05 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139265+0000", 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005767342 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139339+0000", 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4522000000000002e-05 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.829 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:136 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'94, mlcod=14'85)", 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.133248+0000", 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.634651123999999, 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061742519999999999, 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133248+0000", 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133248+0000", 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133249+0000", 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.753e-06 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133262+0000", 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2123000000000001e-05 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133291+0000", 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9575999999999999e-05 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.133304+0000", 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3190000000000001e-05 2024-09-21T23:15:02.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134496+0000", 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001191941 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134552+0000", 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5875999999999998e-05 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139346+0000", 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047937129999999998 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139422+0000", 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6080000000000003e-05 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:137 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'95, mlcod=14'85)", 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.134239+0000", 2024-09-21T23:15:02.834 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.633659742000001, 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007144114, 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134239+0000", 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134239+0000", 2024-09-21T23:15:02.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134241+0000", 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4530000000000001e-06 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134313+0000", 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1116000000000001e-05 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134366+0000", 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3545999999999999e-05 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.134387+0000", 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0497e-05 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.135370+0000", 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098330500000000007 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.135429+0000", 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9437e-05 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141276+0000", 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058461370000000004 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141383+0000", 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107623 2024-09-21T23:15:02.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:138 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'96, mlcod=14'85)", 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.135928+0000", 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.631970479, 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055632069999999997, 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.135928+0000", 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.135928+0000", 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.135930+0000", 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.135945+0000", 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5150000000000001e-05 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.135998+0000", 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2636000000000002e-05 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136013+0000", 2024-09-21T23:15:02.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4927e-05 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136865+0000", 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085183700000000004 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136920+0000", 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5432000000000001e-05 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141387+0000", 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044671490000000001 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141491+0000", 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104196 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:139 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'97, mlcod=14'85)", 2024-09-21T23:15:02.843 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.136562+0000", 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.631336567, 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050356180000000004, 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136562+0000", 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136562+0000", 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136564+0000", 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136576+0000", 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2176e-05 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136600+0000", 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.429e-05 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.136617+0000", 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6350000000000001e-05 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137875+0000", 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012579970000000001 2024-09-21T23:15:02.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137937+0000", 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2195999999999999e-05 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141499+0000", 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035618849999999999 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141598+0000", 2024-09-21T23:15:02.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8987000000000006e-05 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:140 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'98, mlcod=14'85)", 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.137134+0000", 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.630764972, 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045536650000000001, 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137134+0000", 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137134+0000", 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137135+0000", 2024-09-21T23:15:02.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.24e-06 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137287+0000", 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151829 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137307+0000", 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0060000000000001e-05 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.137318+0000", 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1257000000000001e-05 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139508+0000", 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021898289999999999 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.139539+0000", 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1229999999999997e-05 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141601+0000", 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020616179999999999 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.141687+0000", 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6601999999999995e-05 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:141 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'99, mlcod=14'85)", 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.138577+0000", 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.62932163, 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036973449999999999, 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138577+0000", 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138577+0000", 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138579+0000", 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.79e-06 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138591+0000", 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2357e-05 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6373e-05 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.138631+0000", 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3324e-05 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.140384+0000", 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017528789999999999 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.140433+0000", 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9233000000000001e-05 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.142204+0000", 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017713 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.142274+0000", 2024-09-21T23:15:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0088999999999999e-05 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4177.0:142 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'100, mlcod=14'98)", 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:47.143394+0000", 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.62450436, 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026904530000000002, 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:02.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143394+0000", 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143394+0000", 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143396+0000", 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.883e-06 2024-09-21T23:15:02.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143453+0000", 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6746000000000002e-05 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143483+0000", 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0107000000000002e-05 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143497+0000", 2024-09-21T23:15:02.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3540000000000001e-05 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143574+0000", 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6846000000000003e-05 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.143603+0000", 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9756000000000001e-05 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.145953+0000", 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023493289999999998 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:47.146085+0000", 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013224600000000001 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:02.861 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:02.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:02.863 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:02.860+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:02.964 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:02.960+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:03.036 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:03.036 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:03.036 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:03.036 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:03.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:03.064 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:03.059+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:03.165 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:03.164+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:03.215 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:03.215 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:03.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:03.267 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:03.259+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:03.315 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:15:03.367 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:03.363+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:03.406 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:03.406 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:03.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:03.406 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:03.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.406 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4203.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.145227624, 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018542260000000001, 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262586+0000", 2024-09-21T23:15:03.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7399999999999993e-06 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262616+0000", 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0233e-05 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262637+0000", 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0665999999999998e-05 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262764+0000", 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127272 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262823+0000", 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8457e-05 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.264355+0000", 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015325510000000001 2024-09-21T23:15:03.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.264431+0000", 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5753000000000005e-05 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4203.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.137651505000001, 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002054284, 2024-09-21T23:15:03.411 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.13e-07 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270162+0000", 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7870000000000007e-06 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270188+0000", 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6316e-05 2024-09-21T23:15:03.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270216+0000", 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7586e-05 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270366+0000", 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149992 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:03.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270424+0000", 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7979999999999997e-05 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.272120+0000", 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001696227 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.272207+0000", 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6883000000000001e-05 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:03.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4337.0:7 3.1a e23/21 3:5cea7035:::rbd_info:head v 23'1, mlcod=0'0)", 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:02.315865+0000", 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.091939164, 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020428379999999999, 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315865+0000", 2024-09-21T23:15:03.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315865+0000", 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315866+0000", 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315875+0000", 2024-09-21T23:15:03.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.969e-06 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315911+0000", 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6124000000000002e-05 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315944+0000", 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3142999999999998e-05 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:03.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.316135+0000", 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019103800000000001 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.316185+0000", 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0356999999999999e-05 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.317791+0000", 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016055209999999999 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.317908+0000", 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117136 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:03.420 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:03.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:03.468 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:03.463+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:03.550 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:03.551 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:03.551 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:03.551 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:03.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:03.569 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:03.563+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:03.669 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:03.668+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:03.692 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:03.693 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:03.707 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:03.793 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:03.792+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:03.870 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:03.868+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:03.873 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:03.873 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:03.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:03.873 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4203.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.614077149, 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001055961, 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:03.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:03.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261290+0000", 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5509999999999999e-06 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261302+0000", 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2459e-05 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261313+0000", 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521e-05 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261393+0000", 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9802000000000006e-05 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261439+0000", 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6140000000000002e-05 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.262268+0000", 2024-09-21T23:15:03.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000828939 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.262340+0000", 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2108999999999997e-05 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:03.878 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:03.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:03.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:03.972 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:03.968+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:04.018 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:04.018 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:04.018 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:04.018 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:04.025 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:15:04.026 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:15:04.032 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:04.043 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:04.043 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:04.061 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:04.061 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:04.073 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:04.072+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:04.080 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:04.081 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:04.098 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:04.098 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:04.116 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:04.117 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:04.156 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:04.157 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:04.171 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:04.173 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:04.167+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:04.274 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:04.272+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:04.286 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:04.286 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:04.300 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:04.375 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:04.371+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:04.418 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:04.418 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:04.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:04.476 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:04.472+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:04.576 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:04.576+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:04.596 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:04.596 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:04.609 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:15:04.609 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:15:04.610 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-21T23:15:04.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-21T23:15:04.611 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:04.677 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:04.671+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:04.735 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:04.736 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:04.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:04.795 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:04.791+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:04.878 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:04.871+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:04.918 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:04.918 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:04.932 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:04.934 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-21T23:15:04.932+0000 7faab395a640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-09-21T23:15:04.979 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:04.976+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:05.080 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:05.076+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:05.098 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:05.098 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:05.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:05.181 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:05.180+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:05.233 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:05.233 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:05.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:15:05.293 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:05.292+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:05.337 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.3. 2024-09-21T23:15:05.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:15:05.399 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:05.399 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:05.399 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:05.396+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:05.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:15:05.426 INFO:teuthology.orchestra.run.smithi170.stdout:1 write 0x5f8b800 thru 0x5f8ed68 (0x3569 bytes) 2024-09-21T23:15:05.427 INFO:teuthology.orchestra.run.smithi170.stdout:2 read 0x1573800 thru 0x157898c (0x518d bytes) 2024-09-21T23:15:05.455 INFO:teuthology.orchestra.run.smithi170.stdout:3 write 0x7abe800 thru 0x7acc914 (0xe115 bytes) 2024-09-21T23:15:05.457 INFO:teuthology.orchestra.run.smithi170.stdout:truncating to largest ever: 0xf3a600 2024-09-21T23:15:05.469 INFO:teuthology.orchestra.run.smithi170.stdout:8 trunc from 0x7acca00 to 0xf3a600 2024-09-21T23:15:05.483 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:05.479+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:05.575 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:05.575 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:05.584 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:05.579+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:05.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:15:05.750 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:05.748+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:05.849 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:05.847+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:05.925 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:05.925 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:05.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:05.989 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:05.988+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:06.050 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:06.043+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:06.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:06.134 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:06.151 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:06.147+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:06.252 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:06.248+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:06.253 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:06.253 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:06.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:06.310 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:15:06.311 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:15:06.328 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:15:06.329 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:06.329 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:06.355 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:06.347+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:06.357 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:06.358 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:06.380 INFO:teuthology.orchestra.run.smithi170.stdout:11 read 0xe99800 thru 0xe9b2cc (0x1acd bytes) 2024-09-21T23:15:06.383 INFO:teuthology.orchestra.run.smithi170.stdout:13 punch from 0x158000 to 0x15e200, (0x6200 bytes) 2024-09-21T23:15:06.392 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:06.392 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:06.414 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:06.414 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:06.419 INFO:teuthology.orchestra.run.smithi170.stdout:15 write 0x884b200 thru 0x88529dc (0x77dd bytes) 2024-09-21T23:15:06.421 INFO:teuthology.orchestra.run.smithi170.stdout:17 write 0xe73200 thru 0xe7856d (0x536e bytes) 2024-09-21T23:15:06.434 INFO:teuthology.orchestra.run.smithi170.stdout:18 trunc from 0x8852a00 to 0xd91c00 2024-09-21T23:15:06.436 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:06.437 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4203.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186930577, 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018542260000000001, 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.448 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262577+0000", 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262586+0000", 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7399999999999993e-06 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262616+0000", 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0233e-05 2024-09-21T23:15:06.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262637+0000", 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0665999999999998e-05 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262764+0000", 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127272 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.262823+0000", 2024-09-21T23:15:06.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8457e-05 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.264355+0000", 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015325510000000001 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:52.264431+0000", 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5753000000000005e-05 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4203.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.179354458000001, 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002054284, 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270153+0000", 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.13e-07 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270162+0000", 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7870000000000007e-06 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270188+0000", 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6316e-05 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270216+0000", 2024-09-21T23:15:06.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7586e-05 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270366+0000", 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149992 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.270424+0000", 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7979999999999997e-05 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.272120+0000", 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001696227 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:53.272207+0000", 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6883000000000001e-05 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.457 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4337.0:7 3.1a e23/21 3:5cea7035:::rbd_info:head v 23'1, mlcod=0'0)", 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:02.315865+0000", 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.1336421169999999, 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020428379999999999, 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315865+0000", 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315865+0000", 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315866+0000", 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315875+0000", 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.969e-06 2024-09-21T23:15:06.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315911+0000", 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6124000000000002e-05 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.315944+0000", 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3142999999999998e-05 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.316135+0000", 2024-09-21T23:15:06.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019103800000000001 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.316185+0000", 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0356999999999999e-05 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.317791+0000", 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016055209999999999 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:02.317908+0000", 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117136 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4337.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:03.329423+0000", 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.120084265, 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001091678, 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.462 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329423+0000", 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329423+0000", 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329423+0000", 2024-09-21T23:15:06.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0499999999999999e-07 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329431+0000", 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0139999999999998e-06 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329453+0000", 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1297999999999998e-05 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329468+0000", 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5169e-05 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329533+0000", 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5285999999999998e-05 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.329554+0000", 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1268999999999999e-05 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.330474+0000", 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091997699999999999 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:03.330515+0000", 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.036e-05 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.466 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:06.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:06.469 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:06.463+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:06.470 INFO:teuthology.orchestra.run.smithi170.stdout:19 read 0x2ad000 thru 0x2b6c0b (0x9c0c bytes) 2024-09-21T23:15:06.473 INFO:teuthology.orchestra.run.smithi170.stdout:21 read 0x9a9000 thru 0x9ab313 (0x2314 bytes) 2024-09-21T23:15:06.473 INFO:teuthology.orchestra.run.smithi170.stdout:22 read 0x1aca00 thru 0x1b760d (0xac0e bytes) 2024-09-21T23:15:06.499 INFO:teuthology.orchestra.run.smithi170.stdout:23 write 0x3a02800 thru 0x3a0d9f4 (0xb1f5 bytes) 2024-09-21T23:15:06.500 INFO:teuthology.orchestra.run.smithi170.stdout:24 read 0x28e7800 thru 0x28f1400 (0x9c01 bytes) 2024-09-21T23:15:06.508 INFO:teuthology.orchestra.run.smithi170.stdout:25 trunc from 0x3a0da00 to 0x4b9a00 2024-09-21T23:15:06.543 INFO:teuthology.orchestra.run.smithi170.stdout:26 read 0x387200 thru 0x38c036 (0x4e37 bytes) 2024-09-21T23:15:06.546 INFO:teuthology.orchestra.run.smithi170.stdout:27 punch from 0x3fc00 to 0x47000, (0x7400 bytes) 2024-09-21T23:15:06.547 INFO:teuthology.orchestra.run.smithi170.stdout:30 punch from 0x277000 to 0x27c600, (0x5600 bytes) 2024-09-21T23:15:06.547 INFO:teuthology.orchestra.run.smithi170.stdout:31 read 0x402800 thru 0x41138c (0xeb8d bytes) 2024-09-21T23:15:06.547 INFO:teuthology.orchestra.run.smithi170.stdout:34 read 0xf9600 thru 0x102f76 (0x9977 bytes) 2024-09-21T23:15:06.553 INFO:teuthology.orchestra.run.smithi170.stdout:35 punch from 0x24e00 to 0x2ca00, (0x7c00 bytes) 2024-09-21T23:15:06.553 INFO:teuthology.orchestra.run.smithi170.stdout:truncating to largest ever: 0x43b1e00 2024-09-21T23:15:06.562 INFO:teuthology.orchestra.run.smithi170.stdout:37 trunc from 0x4b9a00 to 0x43b1e00 2024-09-21T23:15:06.569 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:06.568+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:truncating to largest ever: 0xa1bc200 2024-09-21T23:15:06.588 INFO:teuthology.orchestra.run.smithi170.stdout:38 trunc from 0x43b1e00 to 0xa1bc200 2024-09-21T23:15:06.604 INFO:teuthology.orchestra.run.smithi170.stdout:40 read 0x2782c00 thru 0x278a258 (0x7659 bytes) 2024-09-21T23:15:06.605 INFO:teuthology.orchestra.run.smithi170.stdout:41 clone 1 order 19 su 2048 sc 3 2024-09-21T23:15:06.647 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:06.647 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:06.647 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:06.647 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:06.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:06.705 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:06.704+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:06.771 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:06.767+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:06.832 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:06.832 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:06.832 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:06.832 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:06.832 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:06.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:06.893 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:06.892+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:06.973 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:06.972+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:06.975 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:06.975 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:06.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4203.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.714334136, 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001055961, 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261284+0000", 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261290+0000", 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5509999999999999e-06 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261302+0000", 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2459e-05 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261313+0000", 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521e-05 2024-09-21T23:15:06.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261393+0000", 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9802000000000006e-05 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.261439+0000", 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6140000000000002e-05 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.262268+0000", 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000828939 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:14:51.262340+0000", 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2108999999999997e-05 2024-09-21T23:15:06.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:120 3.1e 3:7b312d07:::rbd_data.10f16b870c42.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.365629+0000", 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.60998900300000003, 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024484730000000001, 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:06.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 120 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365629+0000", 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365629+0000", 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365632+0000", 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5119999999999998e-06 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365641+0000", 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6879999999999994e-06 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365671+0000", 2024-09-21T23:15:06.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9632e-05 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365685+0000", 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4034e-05 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365786+0000", 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100999 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.365897+0000", 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111196 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.366045+0000", 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014832300000000001 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.366971+0000", 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092526299999999995 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367983+0000", 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001012621 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.368035+0000", 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1371e-05 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.368078+0000", 2024-09-21T23:15:06.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2833999999999997e-05 2024-09-21T23:15:06.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.987 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.987 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4370.0:120 3.1e e26/25 ondisk, result = 0)", 2024-09-21T23:15:06.987 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.367886+0000", 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.60773171500000001, 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019764500000000001, 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367886+0000", 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367886+0000", 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367887+0000", 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043e-06 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367892+0000", 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6009999999999997e-06 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367904+0000", 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2036e-05 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367913+0000", 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367968+0000", 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5648e-05 2024-09-21T23:15:06.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.367977+0000", 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.368084+0000", 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107077 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:144 3.13 e26/25 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:head v 26'1, mlcod=0'0)", 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.390638+0000", 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.58498046999999997, 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001074628, 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390638+0000", 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390638+0000", 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390638+0000", 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7e-07 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390774+0000", 2024-09-21T23:15:06.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013652400000000001 2024-09-21T23:15:06.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390790+0000", 2024-09-21T23:15:06.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5441e-05 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390801+0000", 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.08e-05 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390865+0000", 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4092999999999998e-05 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.390881+0000", 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6526000000000001e-05 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.391643+0000", 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076161000000000002 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.391712+0000", 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9264000000000006e-05 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:193 3.d 3:b6d65572:::rbd_data.10f16b870c42.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write 307200~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.442793+0000", 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.53282553399999999, 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032335770000000001, 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:06.997 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 193 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.442793+0000", 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.442793+0000", 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.442796+0000", 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1999999999999999e-06 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.442917+0000", 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000121662 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.442951+0000", 2024-09-21T23:15:06.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3837000000000001e-05 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.442970+0000", 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9202999999999999e-05 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.443057+0000", 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6533000000000007e-05 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.443220+0000", 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016329800000000001 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.443395+0000", 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000174871 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:07.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.444501+0000", 2024-09-21T23:15:07.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011056569999999999 2024-09-21T23:15:07.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445891+0000", 2024-09-21T23:15:07.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013901849999999999 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445951+0000", 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0461999999999998e-05 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.446026+0000", 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4668999999999999e-05 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4370.0:193 3.d e26/25 ondisk, result = 0)", 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.445790+0000", 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.52982832599999996, 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024490699999999999, 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445790+0000", 2024-09-21T23:15:07.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445790+0000", 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445791+0000", 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.17e-06 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445800+0000", 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7110000000000001e-06 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445812+0000", 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2128e-05 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445819+0000", 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9210000000000003e-06 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445873+0000", 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4132999999999998e-05 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.445885+0000", 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1727e-05 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.446035+0000", 2024-09-21T23:15:07.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015011700000000001 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:204 3.d 3:b6d65572:::rbd_data.10f16b870c42.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.453413+0000", 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.52220537600000005, 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021767650000000002, 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 204 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453413+0000", 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453413+0000", 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453417+0000", 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.143e-06 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453432+0000", 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.52e-05 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453474+0000", 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1977000000000001e-05 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453497+0000", 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3280000000000001e-05 2024-09-21T23:15:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453603+0000", 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106106 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453694+0000", 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0938000000000002e-05 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.453825+0000", 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013065799999999999 2024-09-21T23:15:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.454566+0000", 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000741144 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455525+0000", 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095936899999999996 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455560+0000", 2024-09-21T23:15:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4839000000000003e-05 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455589+0000", 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9111e-05 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4370.0:204 3.d e26/25 ondisk, result = 0)", 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.455431+0000", 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.52018730899999999, 2024-09-21T23:15:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016340300000000001, 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455431+0000", 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455431+0000", 2024-09-21T23:15:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455432+0000", 2024-09-21T23:15:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2359999999999999e-06 2024-09-21T23:15:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455437+0000", 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1839999999999998e-06 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455452+0000", 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5067e-05 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455467+0000", 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5228999999999999e-05 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455513+0000", 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5667999999999997e-05 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455519+0000", 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9379999999999997e-06 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.455594+0000", 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5080999999999995e-05 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:319 3.13 e26/25 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:head v 26'2, mlcod=0'0)", 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.550382+0000", 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.42523621499999997, 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001245194, 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550382+0000", 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550382+0000", 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550383+0000", 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2279999999999999e-06 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550485+0000", 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101585 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550501+0000", 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5954e-05 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550508+0000", 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9550000000000003e-06 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550581+0000", 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.326e-05 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.550605+0000", 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4428000000000001e-05 2024-09-21T23:15:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551573+0000", 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096783000000000004 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551627+0000", 2024-09-21T23:15:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3953999999999999e-05 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:320 3.13 e26/25 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:head v 26'3, mlcod=0'0)", 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.551962+0000", 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.42365638999999999, 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010042759999999999, 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551962+0000", 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551962+0000", 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551962+0000", 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0299999999999999e-07 2024-09-21T23:15:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551983+0000", 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.039e-05 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551992+0000", 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9069999999999996e-06 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.551997+0000", 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.293e-06 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.552049+0000", 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1839000000000003e-05 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.552081+0000", 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2233000000000001e-05 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.552842+0000", 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000760858 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.552966+0000", 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124153 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:324 3.13 e26/25 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:head v 26'4, mlcod=26'2)", 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.556549+0000", 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.41906884999999999, 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010394219999999999, 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556549+0000", 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556549+0000", 2024-09-21T23:15:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556550+0000", 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3099999999999997e-07 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556568+0000", 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8564999999999999e-05 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556579+0000", 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-05 2024-09-21T23:15:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556584+0000", 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3480000000000003e-06 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556631+0000", 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6524999999999999e-05 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.556653+0000", 2024-09-21T23:15:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2002999999999999e-05 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.557468+0000", 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081500799999999999 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.557589+0000", 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012068199999999999 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:07.034 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:07.035 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:07.079 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:07.075+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:07.166 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:07.166 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:07.174 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:07.172+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:07.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:07.276 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:07.272+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:07.350 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:07.351 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:07.351 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:07.351 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:07.351 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:07.366 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:07.376 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:07.376+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:07.477 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:07.471+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:07.494 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:07.494 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:07.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:07.494 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:07.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.269818756999999, 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.495 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.501 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.757413474, 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:07.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.507 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.757348571000001, 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:07.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:07.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:07.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:07.513 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.757202132, 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:07.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.757133017000001, 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.519 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:07.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:07.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.756520691999999, 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:07.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:07.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:07.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.756316287000001, 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:07.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.536 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.756273500999999, 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:07.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:07.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.365315212999999, 2024-09-21T23:15:07.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:07.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:07.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:07.548 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.365228703, 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:07.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.365151814000001, 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.555 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:07.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:07.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364946973999999, 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:07.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:07.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364858691999999, 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364727863999999, 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:07.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:07.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:07.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:07.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364682211000002, 2024-09-21T23:15:07.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:07.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364643783999998, 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.584 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:07.589 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364514322000002, 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:07.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:07.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:07.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364492680000001, 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:07.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:07.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:07.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:07.601 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364472635999999, 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:07.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:07.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.364388644000002, 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:07.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:07.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:07.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.613 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:07.613 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:07.613 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:07.613 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:07.615 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:07.612+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:07.716 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:07.712+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:07.742 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:07.742 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:07.742 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:07.742 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:07.757 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:07.816 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:07.811+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:07.882 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:07.882 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:07.896 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:07.914 INFO:teuthology.orchestra.run.smithi170.stdout:43 read 0x99d6c00 thru 0x99e098d (0x9d8e bytes) 2024-09-21T23:15:07.915 INFO:teuthology.orchestra.run.smithi170.stdout:44 read 0x7d96e00 thru 0x7d97ed6 (0x10d7 bytes) 2024-09-21T23:15:07.916 INFO:teuthology.orchestra.run.smithi170.stdout:45 write 0x55bee00 thru 0x55ca99d (0xbb9e bytes) 2024-09-21T23:15:07.917 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:07.916+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:07.956 INFO:teuthology.orchestra.run.smithi170.stdout:46 trunc from 0xa1bc200 to 0x3540600 2024-09-21T23:15:08.019 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:08.016+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:08.032 INFO:teuthology.orchestra.run.smithi170.stdout:48 punch from 0x179d200 to 0x17ac800, (0xf600 bytes) 2024-09-21T23:15:08.074 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:08.074 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:08.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:02.307304+0000", 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.7620225769999998, 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073089369999999997, 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.075 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 7 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307304+0000", 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307304+0000", 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307307+0000", 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1550000000000001e-06 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307318+0000", 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1813e-05 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307340+0000", 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1772000000000001e-05 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307351+0000", 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1236000000000001e-05 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307413+0000", 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1692999999999994e-05 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307418+0000", 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:15:08.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.311777+0000", 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043591710000000002 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.311893+0000", 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000116267 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.312014+0000", 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012133099999999999 2024-09-21T23:15:08.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.312713+0000", 2024-09-21T23:15:08.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00069857299999999997 2024-09-21T23:15:08.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.314506+0000", 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001792884 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.314551+0000", 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4879999999999997e-05 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.314613+0000", 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2485999999999998e-05 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.082 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.7495505390000003, 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076817589999999998, 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 9 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319781+0000", 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3529999999999997e-06 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319821+0000", 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0571999999999997e-05 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319871+0000", 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0114999999999997e-05 2024-09-21T23:15:08.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319900+0000", 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8802999999999998e-05 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320073+0000", 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000172669 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320090+0000", 2024-09-21T23:15:08.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7223000000000001e-05 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325068+0000", 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049774889999999999 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325233+0000", 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016537600000000001 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:08.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325491+0000", 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025775700000000001 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.326967+0000", 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001476684 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327236+0000", 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026841300000000001 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327389+0000", 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0001528 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327458+0000", 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9505000000000006e-05 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4370.0:192 3.11 3:8d32b2ff:::rbd_data.10f16b870c42.0000000000000003:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2568192~24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:06.438409+0000", 2024-09-21T23:15:08.089 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 1.630917682, 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0031925780000000002, 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4370", 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 192 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.438409+0000", 2024-09-21T23:15:08.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.438409+0000", 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.438412+0000", 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3659999999999999e-06 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.438549+0000", 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013694099999999999 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.438594+0000", 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5411999999999998e-05 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.438617+0000", 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3016000000000001e-05 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.438787+0000", 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169996 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.439115+0000", 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032758199999999998 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.439332+0000", 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021755400000000001 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.441062+0000", 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001729944 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.441346+0000", 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028373299999999999 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.441494+0000", 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014838000000000001 2024-09-21T23:15:08.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:06.441602+0000", 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107654 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:223 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.417079+0000", 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.65224804999999997, 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027510080000000001, 2024-09-21T23:15:08.095 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 223 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417079+0000", 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417079+0000", 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417081+0000", 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417091+0000", 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0276e-05 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417117+0000", 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6078999999999999e-05 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417131+0000", 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3627e-05 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417202+0000", 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0873999999999999e-05 2024-09-21T23:15:08.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417304+0000", 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000102218 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.417461+0000", 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156954 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.418171+0000", 2024-09-21T23:15:08.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00071054199999999999 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.419729+0000", 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0015577880000000001 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.419777+0000", 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7683999999999999e-05 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.419830+0000", 2024-09-21T23:15:08.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3165999999999999e-05 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:224 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.420211+0000", 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.64911615300000003, 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028196300000000001, 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 224 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420211+0000", 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420211+0000", 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420213+0000", 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420221+0000", 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9330000000000007e-06 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420245+0000", 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3062e-05 2024-09-21T23:15:08.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420259+0000", 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4977000000000001e-05 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420313+0000", 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3405000000000001e-05 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420382+0000", 2024-09-21T23:15:08.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9270999999999995e-05 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.420576+0000", 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019407800000000001 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.421185+0000", 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060915199999999996 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.422952+0000", 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001766304 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.422996+0000", 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4274000000000003e-05 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.423030+0000", 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4437000000000002e-05 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:238 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [watch watch cookie 140478314344400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.445376+0000", 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.62395109400000004, 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027384499999999999, 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 238 2024-09-21T23:15:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445376+0000", 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445376+0000", 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445377+0000", 2024-09-21T23:15:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1960000000000001e-06 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445383+0000", 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2870000000000001e-06 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445402+0000", 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9089999999999998e-05 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445414+0000", 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1603000000000001e-05 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445509+0000", 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5027999999999999e-05 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445606+0000", 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7046999999999994e-05 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.445893+0000", 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028681600000000001 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.446853+0000", 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095996000000000005 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448017+0000", 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001164364 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448060+0000", 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2750999999999998e-05 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448114+0000", 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4308000000000002e-05 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:239 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.448506+0000", 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.62082072700000002, 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026746679999999998, 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 239 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448506+0000", 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448506+0000", 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448508+0000", 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.663e-06 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448516+0000", 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.303e-06 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448539+0000", 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.251e-05 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448551+0000", 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2089999999999999e-05 2024-09-21T23:15:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448698+0000", 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014757600000000001 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.448840+0000", 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014169700000000001 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.449250+0000", 2024-09-21T23:15:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00040957599999999998 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.450487+0000", 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001237672 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451022+0000", 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00053510799999999996 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451111+0000", 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8871999999999999e-05 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451181+0000", 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9601e-05 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:240 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.451618+0000", 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.61770888899999998, 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0029043319999999999, 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 240 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451618+0000", 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451618+0000", 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451620+0000", 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1100000000000001e-06 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451630+0000", 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0316000000000001e-05 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451660+0000", 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9869000000000002e-05 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451675+0000", 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4392999999999999e-05 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451769+0000", 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4644999999999999e-05 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.451988+0000", 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000218991 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.452251+0000", 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026284000000000002 2024-09-21T23:15:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.453138+0000", 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00088696800000000002 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.454363+0000", 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012252039999999999 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.454454+0000", 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0200999999999999e-05 2024-09-21T23:15:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.454522+0000", 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8794999999999994e-05 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:248 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [watch unwatch cookie 140478314344400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.463779+0000", 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.60554781300000005, 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026968309999999998, 2024-09-21T23:15:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 248 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.463779+0000", 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.463779+0000", 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.463781+0000", 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9439999999999999e-06 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.463790+0000", 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2760000000000006e-06 2024-09-21T23:15:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.463821+0000", 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1158999999999998e-05 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.463836+0000", 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4636e-05 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.464015+0000", 2024-09-21T23:15:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179326 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.464154+0000", 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138844 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.464381+0000", 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022698 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.465538+0000", 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011564559999999999 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.466280+0000", 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00074178799999999995 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.466372+0000", 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2748e-05 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.466476+0000", 2024-09-21T23:15:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000103674 2024-09-21T23:15:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:281 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [watch watch cookie 140478515651616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.132 INFO:teuthology.orchestra.run.smithi170.stdout:49 write 0xcba8c00 thru 0xcbb72b3 (0xe6b4 bytes) 2024-09-21T23:15:08.132 INFO:teuthology.orchestra.run.smithi170.stdout:51 write 0xc625000 thru 0xc62631f (0x1320 bytes) 2024-09-21T23:15:08.132 INFO:teuthology.orchestra.run.smithi170.stdout:53 read 0xb98a00 thru 0xba4963 (0xbf64 bytes) 2024-09-21T23:15:08.132 INFO:teuthology.orchestra.run.smithi170.stdout:57 read 0x82d8000 thru 0x82e4569 (0xc56a bytes) 2024-09-21T23:15:08.132 INFO:teuthology.orchestra.run.smithi170.stdout:58 read 0xa7e6e00 thru 0xa7f127e (0xa47f bytes) 2024-09-21T23:15:08.132 INFO:teuthology.orchestra.run.smithi170.stdout:60 read 0x39d5600 thru 0x39d678a (0x118b bytes) 2024-09-21T23:15:08.132 INFO:teuthology.orchestra.run.smithi170.stdout:64 trunc from 0xcbb7400 to 0xa13d400 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.757816+0000", 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.31151108399999999, 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027495940000000002, 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 281 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.757816+0000", 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.757816+0000", 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.757817+0000", 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5099999999999999e-06 2024-09-21T23:15:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.757825+0000", 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7519999999999996e-06 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.757840+0000", 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5254e-05 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.757853+0000", 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2411e-05 2024-09-21T23:15:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.758019+0000", 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016654899999999999 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.758177+0000", 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015785300000000001 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.758449+0000", 2024-09-21T23:15:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00027227600000000001 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.759537+0000", 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001087284 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.760332+0000", 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00079502700000000002 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.760435+0000", 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000102804 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.760565+0000", 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013087400000000001 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:25 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [watch watch cookie 139999760990816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.880564+0000", 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.18876311600000001, 2024-09-21T23:15:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002637815, 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 25 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880564+0000", 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880564+0000", 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880565+0000", 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0440000000000001e-06 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880570+0000", 2024-09-21T23:15:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880586+0000", 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6021000000000001e-05 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880594+0000", 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6000000000000001e-06 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880743+0000", 2024-09-21T23:15:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014895500000000001 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.880881+0000", 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013858300000000001 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.881153+0000", 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00027135000000000001 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.882126+0000", 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00097280399999999996 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.883016+0000", 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00089078099999999999 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.883097+0000", 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0280999999999999e-05 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.883202+0000", 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010500700000000001 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:90 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [watch reconnect cookie 139999760990816 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.914278+0000", 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.15504863899999999, 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023159869999999998, 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 90 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914278+0000", 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914278+0000", 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914279+0000", 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914281+0000", 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.08e-06 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914287+0000", 2024-09-21T23:15:08.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1929999999999998e-06 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914291+0000", 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2339999999999996e-06 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914433+0000", 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014136300000000001 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.914574+0000", 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014112 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.916594+0000", 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020205570000000001 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:355 3.9 3:923db98e:::rbd_data.10f16a447f06.00000000000000ac:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 86016~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.968530+0000", 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.100796545, 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026699559999999998, 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 355 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968530+0000", 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968530+0000", 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968532+0000", 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1799999999999999e-06 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968613+0000", 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1555000000000001e-05 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968625+0000", 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1948e-05 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968631+0000", 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.331e-06 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968678+0000", 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6804000000000001e-05 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968780+0000", 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010131799999999999 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.968911+0000", 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000131951 2024-09-21T23:15:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.969833+0000", 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00092154100000000001 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.971074+0000", 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001240772 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.971147+0000", 2024-09-21T23:15:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3004999999999996e-05 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.971200+0000", 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3551000000000003e-05 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:284 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.982552+0000", 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.086774696999999998, 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023359420000000001, 2024-09-21T23:15:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 284 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982552+0000", 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982552+0000", 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982553+0000", 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3399999999999997e-07 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982559+0000", 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4229999999999996e-06 2024-09-21T23:15:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982575+0000", 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5928000000000001e-05 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982584+0000", 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4870000000000008e-06 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982649+0000", 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5497e-05 2024-09-21T23:15:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.982731+0000", 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1855999999999997e-05 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.984888+0000", 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0021568170000000001 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:422 3.9 3:923db98e:::rbd_data.10f16a447f06.00000000000000ac:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.996900+0000", 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.072427410999999997, 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027538609999999998, 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 422 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.996900+0000", 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.996900+0000", 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.996901+0000", 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0410000000000001e-06 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.996906+0000", 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0200000000000002e-06 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.996918+0000", 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2661e-05 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.996927+0000", 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0119999999999994e-06 2024-09-21T23:15:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.996990+0000", 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2514000000000005e-05 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.997054+0000", 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3863999999999997e-05 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.997153+0000", 2024-09-21T23:15:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9245000000000003e-05 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.998304+0000", 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001150709 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.999575+0000", 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001271583 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.999623+0000", 2024-09-21T23:15:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7889999999999997e-05 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.999653+0000", 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0321999999999999e-05 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:292 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.998975+0000", 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.070352320999999995, 2024-09-21T23:15:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023442620000000002, 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 292 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.998975+0000", 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.998975+0000", 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.998976+0000", 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.074e-06 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.998980+0000", 2024-09-21T23:15:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1869999999999999e-06 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.998990+0000", 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8239999999999995e-06 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.998995+0000", 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7100000000000004e-06 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.001140+0000", 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0021449310000000001 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.001211+0000", 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0996999999999998e-05 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.001319+0000", 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107539 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4405.0:438 3.15 e27/25 3:adcc144e:::rbd_data.10f16a447f06.0000000000000069:head v 27'1, mlcod=0'0)", 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.009505+0000", 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.059821891000000002, 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037621709999999999, 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009505+0000", 2024-09-21T23:15:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009505+0000", 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009506+0000", 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.42e-06 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009518+0000", 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1773999999999999e-05 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009540+0000", 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1684000000000001e-05 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009550+0000", 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.03e-05 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009622+0000", 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1871000000000004e-05 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.009653+0000", 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0978000000000001e-05 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.013220+0000", 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003566638 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.013267+0000", 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7506000000000002e-05 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4405.0:440 3.19 e27/21 3:9d7e59e1:::rbd_data.10f16a447f06.000000000000006b:head v 27'1, mlcod=0'0)", 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.014790+0000", 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.054537236000000003, 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0022892009999999998, 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014790+0000", 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014790+0000", 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014791+0000", 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014798+0000", 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8879999999999997e-06 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014812+0000", 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3369e-05 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014821+0000", 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7809999999999994e-06 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014883+0000", 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2470000000000003e-05 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.014904+0000", 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.075e-05 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.017040+0000", 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002135811 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.017079+0000", 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9215000000000003e-05 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:441 3.9 3:958cdb01:::rbd_header.10f16a447f06:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.018157+0000", 2024-09-21T23:15:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.051169766999999998, 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028650479999999998, 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 441 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018157+0000", 2024-09-21T23:15:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018157+0000", 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018159+0000", 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.393e-06 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018165+0000", 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018178+0000", 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3291e-05 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018185+0000", 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3599999999999998e-06 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018241+0000", 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5982e-05 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018321+0000", 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9222999999999997e-05 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.018518+0000", 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019712399999999999 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.020411+0000", 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001893689 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.020925+0000", 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051403500000000003 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.020964+0000", 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9137000000000002e-05 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.021022+0000", 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7717000000000003e-05 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4405.0:485 3.1c e27/21 3:3cf8fa19:::rbd_data.10f16a447f06.000000000000002e:head v 27'5, mlcod=26'3)", 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.040886+0000", 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.028441131000000001, 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028394760000000001, 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.040886+0000", 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.040886+0000", 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.040887+0000", 2024-09-21T23:15:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.606e-06 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.040914+0000", 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7063000000000002e-05 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.040941+0000", 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6302e-05 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.040956+0000", 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5651e-05 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.041026+0000", 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9088e-05 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.041056+0000", 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0627999999999999e-05 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.043670+0000", 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026141889999999998 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.043725+0000", 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4948999999999999e-05 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.187 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:08.188 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:08.188 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:08.128+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:08.189 INFO:teuthology.orchestra.run.smithi170.stdout:66 read 0x4f91800 thru 0x4f9c611 (0xae12 bytes) 2024-09-21T23:15:08.189 INFO:teuthology.orchestra.run.smithi170.stdout:68 write 0xf8ec00 thru 0xf92a39 (0x3e3a bytes) 2024-09-21T23:15:08.189 INFO:teuthology.orchestra.run.smithi170.stdout:69 read 0xfbc800 thru 0xfcaa2f (0xe230 bytes) 2024-09-21T23:15:08.193 INFO:teuthology.orchestra.run.smithi170.stdout:71 write 0xa39e200 thru 0xa3a90d3 (0xaed4 bytes) 2024-09-21T23:15:08.196 INFO:teuthology.orchestra.run.smithi170.stdout:72 trunc from 0xa3a9200 to 0x9beca00 2024-09-21T23:15:08.235 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:08.231+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:08.246 INFO:teuthology.orchestra.run.smithi170.stdout:74 read 0x8e0b000 thru 0x8e19154 (0xe155 bytes) 2024-09-21T23:15:08.247 INFO:teuthology.orchestra.run.smithi170.stdout:77 write 0x64be800 thru 0x64cca87 (0xe288 bytes) 2024-09-21T23:15:08.247 INFO:teuthology.orchestra.run.smithi170.stdout:78 punch from 0x2db0e00 to 0x2db8000, (0x7200 bytes) 2024-09-21T23:15:08.267 INFO:teuthology.orchestra.run.smithi170.stdout:89 trunc from 0x9beca00 to 0x31b1e00 2024-09-21T23:15:08.319 INFO:teuthology.orchestra.run.smithi170.stdout:91 read 0x2c3f200 thru 0x2c4577f (0x6580 bytes) 2024-09-21T23:15:08.326 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:08.327 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:08.327 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:08.327 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:08.333 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:08.332+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:08.341 INFO:teuthology.orchestra.run.smithi170.stdout:92 write 0x5982800 thru 0x598d073 (0xa874 bytes) 2024-09-21T23:15:08.343 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:08.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:15:08.381 INFO:teuthology.orchestra.run.smithi170.stdout:93 trunc from 0x598d200 to 0x11d1c00 2024-09-21T23:15:08.434 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:08.432+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:08.439 INFO:teuthology.orchestra.run.smithi170.stdout:truncating to largest ever: 0xe715c00 2024-09-21T23:15:08.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:08.488 INFO:teuthology.orchestra.run.smithi170.stdout:96 trunc from 0x11d1c00 to 0xe715c00 2024-09-21T23:15:08.503 INFO:teuthology.orchestra.run.smithi170.stdout:99 write 0x2f1e800 thru 0x2f2c1fe (0xd9ff bytes) 2024-09-21T23:15:08.505 INFO:teuthology.orchestra.run.smithi170.stdout:100 punch from 0xd75200 to 0xd84200, (0xf000 bytes) 2024-09-21T23:15:08.514 INFO:teuthology.orchestra.run.smithi170.stdout:102 punch from 0xe050c00 to 0xe051600, (0xa00 bytes) 2024-09-21T23:15:08.514 INFO:teuthology.orchestra.run.smithi170.stdout:105 punch from 0xbe3e200 to 0xbe45400, (0x7200 bytes) 2024-09-21T23:15:08.525 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:08.526 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:08.526 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:08.526 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:08.526 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:08.534 INFO:teuthology.orchestra.run.smithi170.stdout:108 trunc from 0xe715c00 to 0x4b8ac00 2024-09-21T23:15:08.535 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:08.532+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:08.543 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:08.580 INFO:teuthology.orchestra.run.smithi170.stdout:110 read 0x2c5e400 thru 0x2c6ce94 (0xea95 bytes) 2024-09-21T23:15:08.582 INFO:teuthology.orchestra.run.smithi170.stdout:111 write 0xe15200 thru 0xe20cb8 (0xbab9 bytes) 2024-09-21T23:15:08.602 INFO:teuthology.orchestra.run.smithi170.stdout:112 write 0x5e38400 thru 0x5e38b1f (0x720 bytes) 2024-09-21T23:15:08.603 INFO:teuthology.orchestra.run.smithi170.stdout:114 read 0x54b9000 thru 0x54c3781 (0xa782 bytes) 2024-09-21T23:15:08.604 INFO:teuthology.orchestra.run.smithi170.stdout:115 trunc from 0x5e38c00 to 0x572be00 2024-09-21T23:15:08.620 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:15:08.620 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:15:08.635 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:08.632+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:08.638 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:08.639 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:08.656 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:08.656 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:08.671 INFO:teuthology.orchestra.run.smithi170.stdout:116 write 0x6252400 thru 0x625bff1 (0x9bf2 bytes) 2024-09-21T23:15:08.673 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:08.674 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:08.674 INFO:teuthology.orchestra.run.smithi170.stdout:117 read 0x3fcf200 thru 0x3fdec76 (0xfa77 bytes) 2024-09-21T23:15:08.683 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:08.683 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:08.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:08.683 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:08.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.683 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:275 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73071914100000002, 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013739622, 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 275 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949270+0000", 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3120000000000001e-06 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949274+0000", 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9539999999999998e-06 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949277+0000", 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2169999999999999e-06 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.962538+0000", 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013261545 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.962747+0000", 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020921199999999999 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963008+0000", 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026011900000000001 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:276 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73070636099999997, 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014002473, 2024-09-21T23:15:08.688 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 276 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949282+0000", 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.26e-06 2024-09-21T23:15:08.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949286+0000", 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9230000000000004e-06 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949289+0000", 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963078+0000", 2024-09-21T23:15:08.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013788804999999999 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963169+0000", 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1114999999999998e-05 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963283+0000", 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011409 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:277 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73069387699999999, 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014347459999999999, 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 277 2024-09-21T23:15:08.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949294+0000", 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949295+0000", 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2580000000000001e-06 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949299+0000", 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949302+0000", 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9859999999999999e-06 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963339+0000", 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014037304 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963466+0000", 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012664600000000001 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963641+0000", 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017508399999999999 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:278 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73068148399999999, 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014599439000000001, 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.697 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 278 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949307+0000", 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.139e-06 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949311+0000", 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0230000000000003e-06 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949314+0000", 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963697+0000", 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014382544000000001 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963785+0000", 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7587e-05 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963905+0000", 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012056 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.701 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:279 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73066863199999998, 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014999721000000001, 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 279 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949319+0000", 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0600000000000001e-07 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949320+0000", 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.313e-06 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949324+0000", 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9570000000000004e-06 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949327+0000", 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:08.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963958+0000", 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014630341 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964127+0000", 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016937300000000001 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964318+0000", 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019130699999999999 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:280 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.730634797, 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015213915, 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.706 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 280 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:08.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949353+0000", 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949361+0000", 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7709999999999995e-06 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949367+0000", 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.003e-06 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949371+0000", 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964367+0000", 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014995746000000001 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964460+0000", 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3565000000000003e-05 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964566+0000", 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106067 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.710 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:349 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 366592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.71327982300000004, 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013723654, 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 349 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966708+0000", 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8200000000000001e-07 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966710+0000", 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966714+0000", 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.177e-06 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966722+0000", 2024-09-21T23:15:08.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2099999999999993e-06 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980309+0000", 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013586765000000001 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980361+0000", 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1643e-05 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980431+0000", 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0434999999999999e-05 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:350 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.71326051899999998, 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013944434, 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.715 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 350 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.65e-07 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966728+0000", 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3069999999999999e-06 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966732+0000", 2024-09-21T23:15:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9040000000000002e-06 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966735+0000", 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7819999999999998e-06 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980479+0000", 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013744528000000001 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980562+0000", 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2218999999999993e-05 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980671+0000", 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109429 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:351 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.713248416, 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014095582000000001, 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 351 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9799999999999999e-07 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966740+0000", 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966744+0000", 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966747+0000", 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.117e-06 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980710+0000", 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013963074000000001 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980762+0000", 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1779e-05 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980834+0000", 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2538000000000002e-05 2024-09-21T23:15:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:352 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.71320884600000001, 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014313115, 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 352 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966779+0000", 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.22e-07 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966783+0000", 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8099999999999997e-06 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966788+0000", 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.758e-06 2024-09-21T23:15:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966791+0000", 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1489999999999998e-06 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980882+0000", 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014090965 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980969+0000", 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6222000000000002e-05 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981091+0000", 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000122889 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:353 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.71319133800000001, 2024-09-21T23:15:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014443615, 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 353 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966802+0000", 2024-09-21T23:15:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9070000000000004e-06 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966806+0000", 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1200000000000004e-06 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966813+0000", 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5239999999999997e-06 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981114+0000", 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014301385 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981166+0000", 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2021999999999998e-05 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981239+0000", 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3385000000000003e-05 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:972 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.53267574699999998, 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013606012000000001, 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 972 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147312+0000", 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147313+0000", 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.243e-06 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147317+0000", 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.216e-06 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147321+0000", 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160753+0000", 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013432062 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160825+0000", 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.216e-05 2024-09-21T23:15:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160917+0000", 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2522000000000006e-05 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:973 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.53266226299999997, 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013764903, 2024-09-21T23:15:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 973 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8999999999999998e-07 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147327+0000", 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.46e-06 2024-09-21T23:15:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147331+0000", 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2339999999999996e-06 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147334+0000", 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6109999999999998e-06 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160954+0000", 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013619229 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161011+0000", 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7046000000000002e-05 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161090+0000", 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9033e-05 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:974 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 520192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.532648495, 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013957262999999999, 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 974 2024-09-21T23:15:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147340+0000", 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2589999999999999e-06 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147344+0000", 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1899999999999997e-06 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147348+0000", 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161133+0000", 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013784597000000001 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161205+0000", 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2692000000000001e-05 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161296+0000", 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0550000000000005e-05 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2367 3.16 3:6d83b35b:::rbd_object_map.10f16a447f06:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.166637386, 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015679904000000001, 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2367 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513351+0000", 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513355+0000", 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5929999999999999e-06 2024-09-21T23:15:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513372+0000", 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7062999999999999e-05 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513386+0000", 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3293e-05 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523249+0000", 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.009863251 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523437+0000", 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018793 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.524029+0000", 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059259600000000003 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.525598+0000", 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001569017 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528769+0000", 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003170754 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528893+0000", 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012397100000000001 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.529030+0000", 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013660899999999999 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2582 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 366592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.12538532999999999, 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013589129, 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2582 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554603+0000", 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0550000000000001e-06 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554606+0000", 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1429999999999998e-06 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554609+0000", 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6819999999999999e-06 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568014+0000", 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013405525 2024-09-21T23:15:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568092+0000", 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7180000000000003e-05 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568191+0000", 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9331000000000002e-05 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2583 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.12537510099999999, 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013835072, 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2583 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1e-07 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554613+0000", 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.212e-06 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554617+0000", 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554619+0000", 2024-09-21T23:15:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6680000000000001e-06 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568239+0000", 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013620098000000001 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568331+0000", 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1255999999999995e-05 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568447+0000", 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011650399999999999 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2584 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.12536461700000001, 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014048454, 2024-09-21T23:15:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2584 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4999999999999999e-07 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554624+0000", 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.119e-06 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554627+0000", 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1200000000000002e-06 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554630+0000", 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7219999999999999e-06 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568490+0000", 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013860268 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568570+0000", 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9903999999999999e-05 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568671+0000", 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101071 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2585 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.12535423200000001, 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014283607, 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2585 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.77e-07 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554634+0000", 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.144e-06 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554638+0000", 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2440000000000002e-06 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554640+0000", 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6869999999999999e-06 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568717+0000", 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014077118 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568802+0000", 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4079000000000006e-05 2024-09-21T23:15:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568917+0000", 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011495800000000001 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2586 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.125330208, 2024-09-21T23:15:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014464238000000001, 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2586 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.28e-07 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554658+0000", 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.282e-06 2024-09-21T23:15:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554662+0000", 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5470000000000001e-06 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554665+0000", 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568947+0000", 2024-09-21T23:15:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01428159 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569026+0000", 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8950999999999997e-05 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569121+0000", 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5173000000000005e-05 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:08.775 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:08.776 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:08.776 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:08.777 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:08.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:15:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:118 write 0xd5fc800 thru 0xd603c08 (0x7409 bytes) 2024-09-21T23:15:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:119 read 0x46cce00 thru 0x46dbdab (0xefac bytes) 2024-09-21T23:15:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:120 write 0x1ad2e00 thru 0x1ad8d9d (0x5f9e bytes) 2024-09-21T23:15:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:122 write 0x9d90600 thru 0x9d97c3d (0x763e bytes) 2024-09-21T23:15:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:124 read 0x626d800 thru 0x6274624 (0x6e25 bytes) 2024-09-21T23:15:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:125 write 0x90c7c00 thru 0x90d0566 (0x8967 bytes) 2024-09-21T23:15:08.779 INFO:teuthology.orchestra.run.smithi170.stdout:127 read 0xdcaa00 thru 0xdd33a5 (0x89a6 bytes) 2024-09-21T23:15:08.779 INFO:teuthology.orchestra.run.smithi170.stdout:130 write 0x1390e00 thru 0x1396523 (0x5724 bytes) 2024-09-21T23:15:08.779 INFO:teuthology.orchestra.run.smithi170.stdout:132 trunc from 0xd603e00 to 0x65ccc00 2024-09-21T23:15:08.779 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:08.780 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:08.772+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:08.827 INFO:teuthology.orchestra.run.smithi170.stdout:133 write 0xa4a4000 thru 0xa4ab8e1 (0x78e2 bytes) 2024-09-21T23:15:08.830 INFO:teuthology.orchestra.run.smithi170.stdout:134 trunc from 0xa4aba00 to 0x99a5a00 2024-09-21T23:15:08.877 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:08.871+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:08.899 INFO:teuthology.orchestra.run.smithi170.stdout:135 write 0x15f9200 thru 0x1603c1c (0xaa1d bytes) 2024-09-21T23:15:08.923 INFO:teuthology.orchestra.run.smithi170.stdout:136 trunc from 0x99a5a00 to 0x2c0a00 2024-09-21T23:15:08.946 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:08.946 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:08.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:15:08.977 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:08.971+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:09.078 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:09.071+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:09.084 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:09.084 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:09.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:15:09.179 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:09.175+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:09.219 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:09.219 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:09.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:09.281 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:09.275+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:09.287 INFO:teuthology.orchestra.run.smithi170.stdout:138 write 0x320ca00 thru 0x320e792 (0x1d93 bytes) 2024-09-21T23:15:09.335 INFO:teuthology.orchestra.run.smithi170.stdout:139 write 0x5c7c200 thru 0x5c88128 (0xbf29 bytes) 2024-09-21T23:15:09.336 INFO:teuthology.orchestra.run.smithi170.stdout:141 read 0x3749000 thru 0x37516f6 (0x86f7 bytes) 2024-09-21T23:15:09.359 INFO:teuthology.orchestra.run.smithi170.stdout:143 trunc from 0x5c88200 to 0xdf12800 2024-09-21T23:15:09.364 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:15:09.369 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:09.370 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:09.381 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:09.375+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:09.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:09.399 INFO:teuthology.orchestra.run.smithi170.stdout:144 read 0x2adca00 thru 0x2ae744d (0xaa4e bytes) 2024-09-21T23:15:09.401 INFO:teuthology.orchestra.run.smithi170.stdout:145 read 0x39a9e00 thru 0x39b1ae9 (0x7cea bytes) 2024-09-21T23:15:09.402 INFO:teuthology.orchestra.run.smithi170.stdout:147 clone 2 order 19 su 8192 sc 9 2024-09-21T23:15:09.482 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:09.480+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:09.549 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:09.549 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:09.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:09.604 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:09.604+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:09.684 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:09.680+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:09.729 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:09.730 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:09.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:09.789 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:09.788+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:09.886 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:09.879+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:09.915 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:09.916 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:09.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:09.987 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:09.983+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:10.049 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:10.049 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:10.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:10.108 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:10.108+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:10.185 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:10.185 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:10.189 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:10.188+0000 7f325f91e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25181) UID: 0 2024-09-21T23:15:10.199 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:10.291 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:10.283+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:10.331 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:10.331 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:10.344 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:10.367 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T23:15:10.367 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:15:10.367 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-21T23:15:10.367 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T23:15:10.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:15:10.391 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:10.388+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:10.449 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T23:15:10.450 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-21T23:15:10.450 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:15:10.450 DEBUG:teuthology.orchestra.run.smithi053:> type ceph-objectstore-tool 2024-09-21T23:15:10.493 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:10.487+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:10.497 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T23:15:10.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-21T23:15:10.513 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:10.513 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:10.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:10.594 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:10.592+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:10.692 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:10.693 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:10.695 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:10.691+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:10.706 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:10.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:10.829 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:10.828+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:10.872 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:10.873 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:10.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:10.931 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:10.927+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:10.999 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:10.995+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:11.012 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:11.012 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:11.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:11.100 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:11.100+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:11.197 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:11.197 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:11.201 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:11.195+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:11.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:11.302 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:11.295+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:11.345 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:11.346 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:11.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:11.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:15:11.404 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:11.404+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:11.481 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:11.481 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:11.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:11.544 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:11.544+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:11.605 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:11.604+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:11.619 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:11.620 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:11.633 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:11.706 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:11.699+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:11.738 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:15:11.807 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:11.804+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:11.813 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:11.813 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:11.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:11.908 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:11.903+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:11.990 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:11.990 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:12.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:12.009 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:12.003+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:12.110 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:12.103+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:12.122 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:12.122 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:12.137 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:12.211 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:12.207+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:12.269 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:12.270 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:12.284 INFO:teuthology.orchestra.run.smithi053.stdout:3.f 2024-09-21T23:15:12.284 INFO:teuthology.orchestra.run.smithi053.stdout:3.c 2024-09-21T23:15:12.284 INFO:teuthology.orchestra.run.smithi053.stdout:3.b 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.3 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.1f 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.1 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.18 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.17 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.6 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.15 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.12 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.11 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.10 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:2.5 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:3.1b 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:2.0 2024-09-21T23:15:12.285 INFO:teuthology.orchestra.run.smithi053.stdout:2.4 2024-09-21T23:15:12.286 INFO:teuthology.orchestra.run.smithi053.stdout:1.0 2024-09-21T23:15:12.286 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:12.312 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:12.312+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:12.414 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:12.412+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:12.457 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:12.457 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:12.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:12.514 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:12.512+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:12.590 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:12.590 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:12.604 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:12.651 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:12.647+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:12.716 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:12.711+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:12.725 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:12.725 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:12.740 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:12.818 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone1 from 0xdf12800 (overlap 0x2c0a00) to 0xd3600 2024-09-21T23:15:12.827 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:12.823+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:12.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.10 --file /var/log/ceph/exp.3.10.0 2024-09-21T23:15:12.918 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:12.911+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:12.925 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:12.925 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:12.941 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:13.020 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:13.016+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:13.109 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:13.109 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:13.121 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:13.115+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:13.123 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:13.222 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:13.215+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:13.287 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:13.287 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:13.303 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:13.323 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:13.320+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:13.425 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:13.424+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:13.485 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:13.485 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:13.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:13.526 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:13.524+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:13.553 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-21T23:15:13.621 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:13.621 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:13.627 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:13.619+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:13.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:13.727 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:13.724+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:13.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:15:13.779 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:13.779 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:13.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:13.836 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:13.836+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:13.928 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:13.928+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:13.965 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:13.966 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:13.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:14.072 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:14.072+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:14.130 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:14.123+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:14.151 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:14.151 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:14.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:14.257 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:14.256+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:14.328 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:14.328 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:14.331 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:14.327+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:14.342 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:14.432 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:14.427+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:14.494 INFO:teuthology.orchestra.run.smithi170.stdout:151 read 0xcb89600 thru 0xcb8febb (0x68bc bytes) 2024-09-21T23:15:14.495 INFO:teuthology.orchestra.run.smithi170.stdout:152 read 0x80e0200 thru 0x80efc98 (0xfa99 bytes) 2024-09-21T23:15:14.497 INFO:teuthology.orchestra.run.smithi170.stdout:153 read 0x8d7e600 thru 0x8d803ee (0x1def bytes) 2024-09-21T23:15:14.498 INFO:teuthology.orchestra.run.smithi170.stdout:154 write 0x1c69200 thru 0x1c6b921 (0x2722 bytes) 2024-09-21T23:15:14.504 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:14.504 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:14.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:14.564 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:14.564+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:14.575 INFO:teuthology.orchestra.run.smithi170.stdout:155 trunc from 0xdf12800 to 0x1eca00 2024-09-21T23:15:14.634 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:14.632+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:14.650 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:14.650 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:14.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:14.671 INFO:teuthology.orchestra.run.smithi053.stderr:Exporting 3.10 info 3.10( v 28'15 (0'0,28'15] local-lis/les=21/22 n=1 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-21T23:15:14.675 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:0eb5383f:::rbd_data.10f16a447f06.0000000000000005:head# 2024-09-21T23:15:14.675 INFO:teuthology.orchestra.run.smithi053.stderr:Export successful 2024-09-21T23:15:14.735 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:14.732+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:14.754 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:15:14.803 INFO:teuthology.orchestra.run.smithi170.stdout:157 trunc from 0x1eca00 to 0x1d30600 2024-09-21T23:15:14.809 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:14.809 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:14.823 INFO:teuthology.orchestra.run.smithi170.stdout:158 trunc from 0x1d30600 to 0x2b9ee00 2024-09-21T23:15:14.824 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:14.844 INFO:teuthology.orchestra.run.smithi170.stdout:160 punch from 0xb3800 to 0xbe000, (0xa800 bytes) 2024-09-21T23:15:14.846 INFO:teuthology.orchestra.run.smithi170.stdout:164 punch from 0x1dfd600 to 0x1e00a00, (0x3400 bytes) 2024-09-21T23:15:14.850 INFO:teuthology.orchestra.run.smithi170.stdout:167 trunc from 0x2b9ee00 to 0x4208400 2024-09-21T23:15:14.867 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:14.863+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:14.879 INFO:teuthology.orchestra.run.smithi170.stdout:169 punch from 0x37edc00 to 0x37f5600, (0x7a00 bytes) 2024-09-21T23:15:14.899 INFO:teuthology.orchestra.run.smithi170.stdout:170 trunc from 0x4208400 to 0xbfd5a00 2024-09-21T23:15:14.917 INFO:teuthology.orchestra.run.smithi170.stdout:171 read 0xbd47400 thru 0xbd4e967 (0x7568 bytes) 2024-09-21T23:15:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:172 write 0x989aa00 thru 0x989ad14 (0x315 bytes) 2024-09-21T23:15:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:174 punch from 0xa95d600 to 0xa963a00, (0x6400 bytes) 2024-09-21T23:15:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:177 read 0x6572600 thru 0x657e260 (0xbc61 bytes) 2024-09-21T23:15:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:181 punch from 0x72a8200 to 0x72ae000, (0x5e00 bytes) 2024-09-21T23:15:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:182 punch from 0x20ef000 to 0x20fc400, (0xd400 bytes) 2024-09-21T23:15:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:183 read 0x7a52e00 thru 0x7a5326e (0x46f bytes) 2024-09-21T23:15:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:184 write 0x575aa00 thru 0x57683e9 (0xd9ea bytes) 2024-09-21T23:15:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:185 punch from 0x55f2c00 to 0x55f6400, (0x3800 bytes) 2024-09-21T23:15:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:189 punch from 0x1a6e600 to 0x1a7be00, (0xd800 bytes) 2024-09-21T23:15:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:194 read 0x8815000 thru 0x88224e2 (0xd4e3 bytes) 2024-09-21T23:15:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:195 punch from 0x8558800 to 0x8565e00, (0xd600 bytes) 2024-09-21T23:15:14.936 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:14.931+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:14.957 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:14.958 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:14.974 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:15.004 INFO:teuthology.orchestra.run.smithi170.stdout:196 write 0xd2ce800 thru 0xd2cfc3e (0x143f bytes) 2024-09-21T23:15:15.005 INFO:teuthology.orchestra.run.smithi170.stdout:201 punch from 0xf3ce00 to 0xf45800, (0x8a00 bytes) 2024-09-21T23:15:15.006 INFO:teuthology.orchestra.run.smithi170.stdout:204 write 0x5bb0600 thru 0x5bbf009 (0xea0a bytes) 2024-09-21T23:15:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:205 read 0x93d4800 thru 0x93de545 (0x9d46 bytes) 2024-09-21T23:15:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:212 read 0xc128400 thru 0xc132982 (0xa583 bytes) 2024-09-21T23:15:15.008 INFO:teuthology.orchestra.run.smithi170.stdout:213 read 0x48cd600 thru 0x48d1d67 (0x4768 bytes) 2024-09-21T23:15:15.008 INFO:teuthology.orchestra.run.smithi170.stdout:216 write 0xc00ae00 thru 0xc00fd81 (0x4f82 bytes) 2024-09-21T23:15:15.008 INFO:teuthology.orchestra.run.smithi170.stdout:219 punch from 0xc2e4e00 to 0xc2e9a00, (0x4c00 bytes) 2024-09-21T23:15:15.009 INFO:teuthology.orchestra.run.smithi170.stdout:221 read 0x85cbe00 thru 0x85d6e0b (0xb00c bytes) 2024-09-21T23:15:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:224 punch from 0x20d8800 to 0x20dac00, (0x2400 bytes) 2024-09-21T23:15:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:225 punch from 0xb1ab200 to 0xb1b8400, (0xd200 bytes) 2024-09-21T23:15:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:228 read 0xb841400 thru 0xb850c80 (0xf881 bytes) 2024-09-21T23:15:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:230 read 0x8767a00 thru 0x877521f (0xd820 bytes) 2024-09-21T23:15:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:233 read 0xac4000 thru 0xac7b96 (0x3b97 bytes) 2024-09-21T23:15:15.026 INFO:teuthology.orchestra.run.smithi170.stdout:235 trunc from 0xd2cfe00 to 0x67e5800 2024-09-21T23:15:15.063 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:15.059+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:15.073 INFO:teuthology.orchestra.run.smithi170.stdout:236 read 0xeafe00 thru 0xeb4c3d (0x4e3e bytes) 2024-09-21T23:15:15.076 INFO:teuthology.orchestra.run.smithi170.stdout:237 read 0x4ebe600 thru 0x4ecaa2d (0xc42e bytes) 2024-09-21T23:15:15.076 INFO:teuthology.orchestra.run.smithi170.stdout:238 punch from 0x215c200 to 0x2162600, (0x6400 bytes) 2024-09-21T23:15:15.077 INFO:teuthology.orchestra.run.smithi170.stdout:239 write 0x1311a00 thru 0x131c578 (0xab79 bytes) 2024-09-21T23:15:15.077 INFO:teuthology.orchestra.run.smithi170.stdout:243 punch from 0xb9fa00 to 0xba9400, (0x9a00 bytes) 2024-09-21T23:15:15.078 INFO:teuthology.orchestra.run.smithi170.stdout:244 write 0x6752800 thru 0x675bd46 (0x9547 bytes) 2024-09-21T23:15:15.107 INFO:teuthology.orchestra.run.smithi170.stdout:245 write 0xa4e8a00 thru 0xa4f46b1 (0xbcb2 bytes) 2024-09-21T23:15:15.109 INFO:teuthology.orchestra.run.smithi170.stdout:246 write 0x3811600 thru 0x3811972 (0x373 bytes) 2024-09-21T23:15:15.110 INFO:teuthology.orchestra.run.smithi170.stdout:248 read 0xa44c600 thru 0xa456593 (0x9f94 bytes) 2024-09-21T23:15:15.138 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:15.131+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:15.141 INFO:teuthology.orchestra.run.smithi170.stdout:249 write 0xcfbea00 thru 0xcfc13a5 (0x29a6 bytes) 2024-09-21T23:15:15.143 INFO:teuthology.orchestra.run.smithi170.stdout:252 punch from 0x1bf5000 to 0x1bfb000, (0x6000 bytes) 2024-09-21T23:15:15.143 INFO:teuthology.orchestra.run.smithi170.stdout:253 punch from 0xc404600 to 0xc414000, (0xfa00 bytes) 2024-09-21T23:15:15.144 INFO:teuthology.orchestra.run.smithi170.stdout:254 read 0xfa8400 thru 0xfb3d47 (0xb948 bytes) 2024-09-21T23:15:15.145 INFO:teuthology.orchestra.run.smithi170.stdout:257 write 0x7e8e00 thru 0x7f335c (0xa55d bytes) 2024-09-21T23:15:15.155 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:15.156 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:15.172 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:15.176 INFO:teuthology.orchestra.run.smithi170.stdout:258 trunc from 0xcfc1400 to 0x1724000 2024-09-21T23:15:15.226 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.10 2024-09-21T23:15:15.238 INFO:teuthology.orchestra.run.smithi170.stdout:259 read 0x13c7200 thru 0x13cba89 (0x488a bytes) 2024-09-21T23:15:15.242 INFO:teuthology.orchestra.run.smithi170.stdout:263 punch from 0xdc3600 to 0xdcba00, (0x8400 bytes) 2024-09-21T23:15:15.273 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:15.272+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:15.289 INFO:teuthology.orchestra.run.smithi170.stdout:264 write 0xe17ea00 thru 0xe18561b (0x6c1c bytes) 2024-09-21T23:15:15.291 INFO:teuthology.orchestra.run.smithi170.stdout:265 read 0x290e000 thru 0x290fcdf (0x1ce0 bytes) 2024-09-21T23:15:15.291 INFO:teuthology.orchestra.run.smithi170.stdout:266 punch from 0x125e000 to 0x1266e00, (0x8e00 bytes) 2024-09-21T23:15:15.312 INFO:teuthology.orchestra.run.smithi170.stdout:267 trunc from 0xe185800 to 0x1ab7c00 2024-09-21T23:15:15.340 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:15.336+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:15.342 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:15.342 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:15.357 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:15.391 INFO:teuthology.orchestra.run.smithi170.stdout:269 write 0xa0d9c00 thru 0xa0e970c (0xfb0d bytes) 2024-09-21T23:15:15.392 INFO:teuthology.orchestra.run.smithi170.stdout:270 read 0x5039600 thru 0x50468d4 (0xd2d5 bytes) 2024-09-21T23:15:15.393 INFO:teuthology.orchestra.run.smithi170.stdout:271 write 0x5662600 thru 0x5668e3e (0x683f bytes) 2024-09-21T23:15:15.393 INFO:teuthology.orchestra.run.smithi170.stdout:273 write 0x88f0200 thru 0x88f39d1 (0x37d2 bytes) 2024-09-21T23:15:15.393 INFO:teuthology.orchestra.run.smithi170.stdout:275 write 0x5213600 thru 0x5217c8b (0x468c bytes) 2024-09-21T23:15:15.394 INFO:teuthology.orchestra.run.smithi170.stdout:276 punch from 0x7ed4800 to 0x7edc800, (0x8000 bytes) 2024-09-21T23:15:15.441 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:15.440+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:15.455 INFO:teuthology.orchestra.run.smithi170.stdout:278 write 0xa630200 thru 0xa63d246 (0xd047 bytes) 2024-09-21T23:15:15.457 INFO:teuthology.orchestra.run.smithi170.stdout:281 write 0x2627600 thru 0x262770f (0x110 bytes) 2024-09-21T23:15:15.471 INFO:teuthology.orchestra.run.smithi170.stdout:282 trunc from 0xa63d400 to 0xe514e00 2024-09-21T23:15:15.490 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:15.490 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:15.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:15.538 INFO:teuthology.orchestra.run.smithi170.stdout:283 trunc from 0xe514e00 to 0x14d9c00 2024-09-21T23:15:15.542 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:15.540+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:15.595 INFO:teuthology.orchestra.run.smithi170.stdout:286 punch from 0x93de00 to 0x940600, (0x2800 bytes) 2024-09-21T23:15:15.597 INFO:teuthology.orchestra.run.smithi170.stdout:287 read 0x8bc400 thru 0x8be726 (0x2327 bytes) 2024-09-21T23:15:15.597 INFO:teuthology.orchestra.run.smithi170.stdout:288 read 0x721200 thru 0x727b51 (0x6952 bytes) 2024-09-21T23:15:15.634 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:15.634 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:15.636 INFO:teuthology.orchestra.run.smithi170.stdout:289 write 0x91a7800 thru 0x91b6a69 (0xf26a bytes) 2024-09-21T23:15:15.643 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:15.639+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:15.645 INFO:teuthology.orchestra.run.smithi170.stdout:290 trunc from 0x91b6c00 to 0x5a37800 2024-09-21T23:15:15.650 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:15.699 INFO:teuthology.orchestra.run.smithi170.stdout:292 read 0x52c600 thru 0x532e6d (0x686e bytes) 2024-09-21T23:15:15.712 INFO:teuthology.orchestra.run.smithi170.stdout:295 trunc from 0x5a37800 to 0x9fa4800 2024-09-21T23:15:15.733 INFO:teuthology.orchestra.run.smithi170.stdout:296 read 0xd65800 thru 0xd6978e (0x3f8f bytes) 2024-09-21T23:15:15.736 INFO:teuthology.orchestra.run.smithi170.stdout:297 write 0x75db800 thru 0x75e87cd (0xcfce bytes) 2024-09-21T23:15:15.736 INFO:teuthology.orchestra.run.smithi170.stdout:298 read 0x3e77e00 thru 0x3e86ba5 (0xeda6 bytes) 2024-09-21T23:15:15.737 INFO:teuthology.orchestra.run.smithi170.stdout:299 read 0x88bb600 thru 0x88bfa83 (0x4484 bytes) 2024-09-21T23:15:15.744 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:15.739+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:15.753 INFO:teuthology.orchestra.run.smithi170.stdout:301 trunc from 0x9fa4800 to 0x2c84400 2024-09-21T23:15:15.815 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:15.815 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:15.831 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:15.845 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:15.844+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:15.847 INFO:teuthology.orchestra.run.smithi170.stdout:302 write 0xd4b4800 thru 0xd4bd6ef (0x8ef0 bytes) 2024-09-21T23:15:15.849 INFO:teuthology.orchestra.run.smithi170.stdout:304 read 0x9e01e00 thru 0x9e03b44 (0x1d45 bytes) 2024-09-21T23:15:15.875 INFO:teuthology.orchestra.run.smithi170.stdout:305 trunc from 0xd4bd800 to 0x297dc00 2024-09-21T23:15:15.910 INFO:teuthology.orchestra.run.smithi170.stdout:307 trunc from 0x297dc00 to 0x4abe800 2024-09-21T23:15:15.921 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:15.933 INFO:teuthology.orchestra.run.smithi170.stdout:309 trunc from 0x4abe800 to 0x7891200 2024-09-21T23:15:15.944 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:15.944 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:15.946 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:15.944+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:15.946 INFO:teuthology.orchestra.run.smithi170.stdout:316 punch from 0x1741a00 to 0x174b800, (0x9e00 bytes) 2024-09-21T23:15:15.951 INFO:teuthology.orchestra.run.smithi170.stdout:317 trunc from 0x7891200 to 0x9678800 2024-09-21T23:15:15.971 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:15.971 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:15.972 INFO:teuthology.orchestra.run.smithi170.stdout:318 write 0x1c40000 thru 0x1c4b81c (0xb81d bytes) 2024-09-21T23:15:15.972 INFO:teuthology.orchestra.run.smithi170.stdout:319 read 0x7097e00 thru 0x70a270d (0xa90e bytes) 2024-09-21T23:15:15.972 INFO:teuthology.orchestra.run.smithi170.stdout:321 punch from 0x37e6400 to 0x37e8800, (0x2400 bytes) 2024-09-21T23:15:15.977 INFO:teuthology.orchestra.run.smithi170.stdout:322 trunc from 0x9678800 to 0x6435000 2024-09-21T23:15:15.995 INFO:teuthology.orchestra.run.smithi170.stdout:323 punch from 0x15b6800 to 0x15ba000, (0x3800 bytes) 2024-09-21T23:15:15.998 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:15.998 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:16.014 INFO:teuthology.orchestra.run.smithi170.stdout:324 write 0xb487400 thru 0xb4876ae (0x2af bytes) 2024-09-21T23:15:16.015 INFO:teuthology.orchestra.run.smithi170.stdout:325 write 0x8370c00 thru 0x837b72b (0xab2c bytes) 2024-09-21T23:15:16.015 INFO:teuthology.orchestra.run.smithi170.stdout:329 read 0x26ca200 thru 0x26d7219 (0xd01a bytes) 2024-09-21T23:15:16.015 INFO:teuthology.orchestra.run.smithi170.stdout:330 write 0x43cda00 thru 0x43d3bcd (0x61ce bytes) 2024-09-21T23:15:16.016 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:16.016 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:16.019 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:16.019 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:16.023 INFO:teuthology.orchestra.run.smithi170.stdout:331 trunc from 0xb487800 to 0x6d56e00 2024-09-21T23:15:16.032 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:16.045 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:16.046 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:16.050 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:16.050 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:16.062 INFO:teuthology.orchestra.run.smithi170.stdout:333 trunc from 0x6d56e00 to 0x2e3a000 2024-09-21T23:15:16.079 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:16.075+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:16.109 INFO:teuthology.orchestra.run.smithi170.stdout:335 write 0xcfa1e00 thru 0xcfa8d17 (0x6f18 bytes) 2024-09-21T23:15:16.133 INFO:teuthology.orchestra.run.smithi170.stdout:338 trunc from 0xcfa8e00 to 0x2ebca00 2024-09-21T23:15:16.147 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:16.143+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:16.162 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:275 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2078625519999999, 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013739622, 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.163 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 275 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:16.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949270+0000", 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3120000000000001e-06 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949274+0000", 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9539999999999998e-06 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949277+0000", 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2169999999999999e-06 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.962538+0000", 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013261545 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.962747+0000", 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020921199999999999 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963008+0000", 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026011900000000001 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.167 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:276 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2078497719999994, 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014002473, 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 276 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:16.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949282+0000", 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.26e-06 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949286+0000", 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9230000000000004e-06 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949289+0000", 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963078+0000", 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013788804999999999 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963169+0000", 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1114999999999998e-05 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963283+0000", 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011409 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:277 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2078372880000003, 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014347459999999999, 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.172 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 277 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949294+0000", 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949295+0000", 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2580000000000001e-06 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949299+0000", 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949302+0000", 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9859999999999999e-06 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963339+0000", 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014037304 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963466+0000", 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012664600000000001 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963641+0000", 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017508399999999999 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.176 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:278 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2078248949999999, 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014599439000000001, 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 278 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949307+0000", 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.139e-06 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949311+0000", 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0230000000000003e-06 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949314+0000", 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963697+0000", 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014382544000000001 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963785+0000", 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7587e-05 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963905+0000", 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012056 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:279 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2078120430000006, 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014999721000000001, 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.181 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 279 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949319+0000", 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0600000000000001e-07 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949320+0000", 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.313e-06 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949324+0000", 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9570000000000004e-06 2024-09-21T23:15:16.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949327+0000", 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963958+0000", 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014630341 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964127+0000", 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016937300000000001 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964318+0000", 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019130699999999999 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.185 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:280 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2077782080000006, 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015213915, 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 280 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.186 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949353+0000", 2024-09-21T23:15:16.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949361+0000", 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7709999999999995e-06 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949367+0000", 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.003e-06 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949371+0000", 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964367+0000", 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014995746000000001 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964460+0000", 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3565000000000003e-05 2024-09-21T23:15:16.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964566+0000", 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106067 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:349 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 366592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.1904232340000007, 2024-09-21T23:15:16.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013723654, 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 349 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966708+0000", 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8200000000000001e-07 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966710+0000", 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966714+0000", 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.177e-06 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966722+0000", 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2099999999999993e-06 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980309+0000", 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013586765000000001 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980361+0000", 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1643e-05 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980431+0000", 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0434999999999999e-05 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:350 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.1904039300000004, 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013944434, 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 350 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.65e-07 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966728+0000", 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3069999999999999e-06 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966732+0000", 2024-09-21T23:15:16.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9040000000000002e-06 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966735+0000", 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7819999999999998e-06 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980479+0000", 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013744528000000001 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980562+0000", 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2218999999999993e-05 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980671+0000", 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109429 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:351 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.1903918269999991, 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014095582000000001, 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 351 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.200 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:16.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9799999999999999e-07 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966740+0000", 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966744+0000", 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966747+0000", 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.117e-06 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980710+0000", 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013963074000000001 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980762+0000", 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1779e-05 2024-09-21T23:15:16.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980834+0000", 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2538000000000002e-05 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:352 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.1903522570000007, 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014313115, 2024-09-21T23:15:16.204 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 352 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966779+0000", 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.22e-07 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966783+0000", 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8099999999999997e-06 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966788+0000", 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.758e-06 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966791+0000", 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1489999999999998e-06 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980882+0000", 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014090965 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980969+0000", 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6222000000000002e-05 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981091+0000", 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000122889 2024-09-21T23:15:16.208 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:353 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.1903347489999998, 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014443615, 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 353 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966802+0000", 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9070000000000004e-06 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966806+0000", 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1200000000000004e-06 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966813+0000", 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5239999999999997e-06 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981114+0000", 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014301385 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981166+0000", 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2021999999999998e-05 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981239+0000", 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3385000000000003e-05 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.213 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:972 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.0098191580000009, 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013606012000000001, 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 972 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147312+0000", 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:16.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147313+0000", 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.243e-06 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147317+0000", 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.216e-06 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147321+0000", 2024-09-21T23:15:16.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160753+0000", 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013432062 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160825+0000", 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.216e-05 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160917+0000", 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2522000000000006e-05 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:973 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.0098056740000008, 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013764903, 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.218 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 973 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8999999999999998e-07 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147327+0000", 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.46e-06 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147331+0000", 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2339999999999996e-06 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147334+0000", 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6109999999999998e-06 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160954+0000", 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013619229 2024-09-21T23:15:16.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161011+0000", 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7046000000000002e-05 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161090+0000", 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9033e-05 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:974 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 520192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.0097919060000002, 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013957262999999999, 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 974 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.223 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:16.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147340+0000", 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2589999999999999e-06 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147344+0000", 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1899999999999997e-06 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147348+0000", 2024-09-21T23:15:16.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161133+0000", 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013784597000000001 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161205+0000", 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2692000000000001e-05 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161296+0000", 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0550000000000005e-05 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2367 3.16 3:6d83b35b:::rbd_object_map.10f16a447f06:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6437807969999998, 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015679904000000001, 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.227 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2367 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513351+0000", 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513355+0000", 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5929999999999999e-06 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513372+0000", 2024-09-21T23:15:16.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7062999999999999e-05 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513386+0000", 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3293e-05 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523249+0000", 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.009863251 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523437+0000", 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018793 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.524029+0000", 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059259600000000003 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.525598+0000", 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001569017 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528769+0000", 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003170754 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528893+0000", 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012397100000000001 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.529030+0000", 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013660899999999999 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.232 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2582 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 366592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6025287410000004, 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013589129, 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2582 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554603+0000", 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0550000000000001e-06 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554606+0000", 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1429999999999998e-06 2024-09-21T23:15:16.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554609+0000", 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6819999999999999e-06 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568014+0000", 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013405525 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568092+0000", 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7180000000000003e-05 2024-09-21T23:15:16.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568191+0000", 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9331000000000002e-05 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2583 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6025185119999996, 2024-09-21T23:15:16.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013835072, 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2583 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1e-07 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554613+0000", 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.212e-06 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554617+0000", 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554619+0000", 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6680000000000001e-06 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568239+0000", 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013620098000000001 2024-09-21T23:15:16.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568331+0000", 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1255999999999995e-05 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568447+0000", 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011650399999999999 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.241 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2584 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6025080279999999, 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014048454, 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2584 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4999999999999999e-07 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554624+0000", 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.119e-06 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554627+0000", 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1200000000000002e-06 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554630+0000", 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7219999999999999e-06 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568490+0000", 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013860268 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568570+0000", 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9903999999999999e-05 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568671+0000", 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101071 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2585 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6024976429999995, 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014283607, 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2585 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.77e-07 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554634+0000", 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.144e-06 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554638+0000", 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2440000000000002e-06 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554640+0000", 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6869999999999999e-06 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568717+0000", 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014077118 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568802+0000", 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4079000000000006e-05 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568917+0000", 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011495800000000001 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2586 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6024736189999995, 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014464238000000001, 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.250 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2586 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:16.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.28e-07 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554658+0000", 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.282e-06 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554662+0000", 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5470000000000001e-06 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554665+0000", 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568947+0000", 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01428159 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569026+0000", 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8950999999999997e-05 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:16.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569121+0000", 2024-09-21T23:15:16.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5173000000000005e-05 2024-09-21T23:15:16.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.254 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:16.254 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:16.254 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:16.255 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:16.252+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:16.256 INFO:teuthology.orchestra.run.smithi170.stdout:339 write 0x532ce00 thru 0x5335c32 (0x8e33 bytes) 2024-09-21T23:15:16.256 INFO:teuthology.orchestra.run.smithi170.stdout:340 punch from 0x53ca00 to 0x546400, (0x9a00 bytes) 2024-09-21T23:15:16.256 INFO:teuthology.orchestra.run.smithi170.stdout:341 trunc from 0x5335e00 to 0xc121000 2024-09-21T23:15:16.256 INFO:teuthology.orchestra.run.smithi170.stdout:344 punch from 0x27bf400 to 0x27caa00, (0xb600 bytes) 2024-09-21T23:15:16.256 INFO:teuthology.orchestra.run.smithi170.stdout:345 write 0xdc44400 thru 0xdc47527 (0x3128 bytes) 2024-09-21T23:15:16.256 INFO:teuthology.orchestra.run.smithi170.stdout:346 trunc from 0xdc47600 to 0x613ea00 2024-09-21T23:15:16.257 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:16.259 INFO:teuthology.orchestra.run.smithi170.stdout:349 write 0x2e94000 thru 0x2ea1766 (0xd767 bytes) 2024-09-21T23:15:16.260 INFO:teuthology.orchestra.run.smithi170.stdout:350 read 0x140600 thru 0x147c7e (0x767f bytes) 2024-09-21T23:15:16.268 INFO:teuthology.orchestra.run.smithi170.stdout:351 trunc from 0x613ea00 to 0x9a6d400 2024-09-21T23:15:16.294 INFO:teuthology.orchestra.run.smithi170.stdout:355 write 0x7018c00 thru 0x701dcb1 (0x50b2 bytes) 2024-09-21T23:15:16.297 INFO:teuthology.orchestra.run.smithi170.stdout:357 read 0x5092000 thru 0x509bb0a (0x9b0b bytes) 2024-09-21T23:15:16.306 INFO:teuthology.orchestra.run.smithi170.stdout:358 trunc from 0x9a6d400 to 0x62ff200 2024-09-21T23:15:16.333 INFO:teuthology.orchestra.run.smithi170.stdout:366 read 0x563f800 thru 0x564e0c3 (0xe8c4 bytes) 2024-09-21T23:15:16.339 INFO:teuthology.orchestra.run.smithi170.stdout:367 trunc from 0x62ff200 to 0x4dca000 2024-09-21T23:15:16.356 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:16.356+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:16.362 INFO:teuthology.orchestra.run.smithi170.stdout:370 read 0x3cfae00 thru 0x3d0481a (0x9a1b bytes) 2024-09-21T23:15:16.391 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:5884 3.7 e34/21 3:e1bff944:::rbd_object_map.10f1e443a6fa:head v 34'324, mlcod=34'322)", 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.392051+0000", 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0012707860000000001, 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001296092, 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392051+0000", 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392051+0000", 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392053+0000", 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.198e-06 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392063+0000", 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0196e-05 2024-09-21T23:15:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392090+0000", 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6840000000000001e-05 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392101+0000", 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1082e-05 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392221+0000", 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000120723 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.392256+0000", 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4106000000000002e-05 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-21T23:15:16.395 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:16.402 INFO:teuthology.orchestra.run.smithi170.stdout:371 write 0xd80cc00 thru 0xd818e8d (0xc28e bytes) 2024-09-21T23:15:16.404 INFO:teuthology.orchestra.run.smithi170.stdout:374 read 0x233b400 thru 0x23487bf (0xd3c0 bytes) 2024-09-21T23:15:16.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:16.420 INFO:teuthology.orchestra.run.smithi170.stdout:375 trunc from 0xd819000 to 0x3a7fa00 2024-09-21T23:15:16.457 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:16.451+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:16.463 INFO:teuthology.orchestra.run.smithi170.stdout:378 read 0x1139a00 thru 0x11492c6 (0xf8c7 bytes) 2024-09-21T23:15:16.466 INFO:teuthology.orchestra.run.smithi170.stdout:380 read 0x19e0c00 thru 0x19e9f62 (0x9363 bytes) 2024-09-21T23:15:16.467 INFO:teuthology.orchestra.run.smithi170.stdout:383 read 0x1bbe00 thru 0x1bcca1 (0xea2 bytes) 2024-09-21T23:15:16.468 INFO:teuthology.orchestra.run.smithi170.stdout:384 punch from 0x379600 to 0x37ce00, (0x3800 bytes) 2024-09-21T23:15:16.474 INFO:teuthology.orchestra.run.smithi170.stdout:388 trunc from 0x3a7fa00 to 0x9f8000 2024-09-21T23:15:16.521 INFO:teuthology.orchestra.run.smithi170.stdout:389 read 0x927e00 thru 0x933a33 (0xbc34 bytes) 2024-09-21T23:15:16.548 INFO:teuthology.orchestra.run.smithi170.stdout:390 write 0x23bd200 thru 0x23caafa (0xd8fb bytes) 2024-09-21T23:15:16.558 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:16.551+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:16.587 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:16.588 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:16.588 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:16.588 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:16.588 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:16.590 INFO:teuthology.orchestra.run.smithi170.stdout:392 write 0x546a000 thru 0x546dc6e (0x3c6f bytes) 2024-09-21T23:15:16.591 INFO:teuthology.orchestra.run.smithi170.stdout:395 read 0x3017000 thru 0x30218e8 (0xa8e9 bytes) 2024-09-21T23:15:16.591 INFO:teuthology.orchestra.run.smithi170.stdout:396 read 0x34ade00 thru 0x34b3e06 (0x6007 bytes) 2024-09-21T23:15:16.591 INFO:teuthology.orchestra.run.smithi170.stdout:397 read 0x30d2000 thru 0x30db28c (0x928d bytes) 2024-09-21T23:15:16.600 INFO:teuthology.orchestra.run.smithi170.stdout:400 trunc from 0x546de00 to 0x199b400 2024-09-21T23:15:16.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:16.640 INFO:teuthology.orchestra.run.smithi170.stdout:401 punch from 0xcb5a00 to 0xcc1800, (0xbe00 bytes) 2024-09-21T23:15:16.645 INFO:teuthology.orchestra.run.smithi170.stdout:405 flatten 2024-09-21T23:15:16.658 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:16.651+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:16.664 INFO:teuthology.orchestra.run.smithi170.stdout:406 read 0x1197000 thru 0x119c54c (0x554d bytes) 2024-09-21T23:15:16.665 INFO:teuthology.orchestra.run.smithi170.stdout:407 punch from 0x78f600 to 0x799c00, (0xa600 bytes) 2024-09-21T23:15:16.689 INFO:teuthology.orchestra.run.smithi170.stdout:408 trunc from 0x199b400 to 0x9d9d200 2024-09-21T23:15:16.702 INFO:teuthology.orchestra.run.smithi170.stdout:409 write 0x4557c00 thru 0x455a67c (0x2a7d bytes) 2024-09-21T23:15:16.705 INFO:teuthology.orchestra.run.smithi170.stdout:410 read 0x297ce00 thru 0x2982d4b (0x5f4c bytes) 2024-09-21T23:15:16.705 INFO:teuthology.orchestra.run.smithi170.stdout:411 punch from 0x367ea00 to 0x368ca00, (0xe000 bytes) 2024-09-21T23:15:16.706 INFO:teuthology.orchestra.run.smithi170.stdout:413 read 0x6d94c00 thru 0x6da47a6 (0xfba7 bytes) 2024-09-21T23:15:16.737 INFO:teuthology.orchestra.run.smithi170.stdout:418 write 0xd5da800 thru 0xd5e9bee (0xf3ef bytes) 2024-09-21T23:15:16.746 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:16.746 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:16.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:16.746 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:16.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:116 3.1e e26/25 3:7b312d07:::rbd_data.10f16b870c42.000000000000001e:head v 26'1, mlcod=0'0)", 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:06.341322+0000", 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.404356100999999, 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031522109999999998, 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341322+0000", 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341322+0000", 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341322+0000", 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9799999999999999e-07 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341390+0000", 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8064000000000004e-05 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341400+0000", 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4080000000000004e-06 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341411+0000", 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0985000000000001e-05 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341906+0000", 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00049533599999999998 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.341943+0000", 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7416999999999998e-05 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.344425+0000", 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024820110000000001 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:06.344474+0000", 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8692000000000001e-05 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.4004813279999997, 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:16.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "MOSDPGPush(1.0 27/26 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 15'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@15'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.366842+0000", 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.3788364319999999, 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027607220000000002, 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.366842+0000", 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.366842+0000", 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.366843+0000", 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.437e-06 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.367316+0000", 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047277899999999999 2024-09-21T23:15:16.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.367373+0000", 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6889999999999999e-05 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.367425+0000", 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1932999999999999e-05 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.367712+0000", 2024-09-21T23:15:16.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028782800000000001 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.367756+0000", 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3396000000000003e-05 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.369602+0000", 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018464589999999999 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:440 3.19 3:9d7e59e1:::rbd_data.10f16a447f06.000000000000006b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.017583+0000", 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7280950090000005, 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038871629999999999, 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 440 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.017583+0000", 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.017583+0000", 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.017586+0000", 2024-09-21T23:15:16.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.017598+0000", 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1887e-05 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.017627+0000", 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.919e-05 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.017642+0000", 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5443000000000001e-05 2024-09-21T23:15:16.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.017759+0000", 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011712199999999999 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.018021+0000", 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000261592 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:16.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.018334+0000", 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031305100000000001 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.019787+0000", 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014528810000000001 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.021178+0000", 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001391066 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.021337+0000", 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015936600000000001 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.021470+0000", 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132805 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:490 3.2 3:40fd9093:::rbd_data.10f16a447f06.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 389120~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.767 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6977116429999999, 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00426217, 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 490 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047969+0000", 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3329999999999999e-06 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048100+0000", 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013150299999999999 2024-09-21T23:15:16.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048122+0000", 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2266e-05 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048136+0000", 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3125999999999999e-05 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048250+0000", 2024-09-21T23:15:16.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114913 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048473+0000", 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022239199999999999 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048789+0000", 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031575700000000001 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.050278+0000", 2024-09-21T23:15:16.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001489282 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052058+0000", 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001780479 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052148+0000", 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9300000000000002e-05 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052228+0000", 2024-09-21T23:15:16.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0819e-05 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:693 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'16, mlcod=27'14)", 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.111729+0000", 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6339489579999995, 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031398089999999999, 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111729+0000", 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111729+0000", 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111731+0000", 2024-09-21T23:15:16.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111743+0000", 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1892999999999999e-05 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111771+0000", 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8853e-05 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111785+0000", 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3497e-05 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111916+0000", 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013085500000000001 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.111953+0000", 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6850000000000001e-05 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114799+0000", 2024-09-21T23:15:16.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028468180000000001 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114869+0000", 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9363000000000007e-05 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:694 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'17, mlcod=27'14)", 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112215+0000", 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6334627590000004, 2024-09-21T23:15:16.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002698579, 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112215+0000", 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112215+0000", 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112216+0000", 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1829999999999999e-06 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112226+0000", 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1740000000000006e-06 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112243+0000", 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7655999999999998e-05 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112253+0000", 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9769999999999995e-06 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112694+0000", 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044058600000000003 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112751+0000", 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7373000000000001e-05 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114872+0000", 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002120827 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114914+0000", 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1802999999999999e-05 2024-09-21T23:15:16.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6328389669999996, 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:16.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:16.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:16.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:696 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'19, mlcod=27'14)", 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6323323740000006, 2024-09-21T23:15:16.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042479270000000003, 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113347+0000", 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113355+0000", 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113373+0000", 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7226000000000001e-05 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113382+0000", 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7999999999999993e-06 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114156+0000", 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077335100000000003 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114193+0000", 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6946999999999997e-05 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117516+0000", 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033230299999999998 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117594+0000", 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7860000000000003e-05 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:698 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'21, mlcod=27'14)", 2024-09-21T23:15:16.790 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.115544+0000", 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6301340950000007, 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003181606, 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.115544+0000", 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.115544+0000", 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.115545+0000", 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8700000000000005e-07 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.115552+0000", 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4800000000000004e-06 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.115566+0000", 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3463e-05 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.115575+0000", 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-06 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.116393+0000", 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081850399999999995 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.116453+0000", 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9935999999999997e-05 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.118600+0000", 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002147337 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.118725+0000", 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012519899999999999 2024-09-21T23:15:16.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:1660 3.19 3:9d7e59e1:::rbd_data.10f16a447f06.000000000000006b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.287802+0000", 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.4578756669999997, 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040976110000000001, 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1660 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.287802+0000", 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.287802+0000", 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.287806+0000", 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3400000000000002e-06 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.287819+0000", 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3658999999999999e-05 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.287865+0000", 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5442999999999998e-05 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.287888+0000", 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3257000000000001e-05 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.288014+0000", 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012645500000000001 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.288144+0000", 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129443 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.288431+0000", 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028716400000000002 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.290220+0000", 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017891300000000001 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.291718+0000", 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001497405 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.291816+0000", 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8222999999999998e-05 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.291900+0000", 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4091999999999994e-05 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:1691 3.0 3:07e4adab:::rbd_data.10f16a447f06.0000000000000063:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 69120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.308047+0000", 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.437631047, 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030325830000000002, 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1691 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308047+0000", 2024-09-21T23:15:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308047+0000", 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308049+0000", 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308061+0000", 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1527e-05 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308096+0000", 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4993000000000002e-05 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308116+0000", 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0562999999999999e-05 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308251+0000", 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013475600000000001 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308494+0000", 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243234 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.308835+0000", 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000340825 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.309939+0000", 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001104418 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.310920+0000", 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098003799999999992 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.311008+0000", 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8852999999999998e-05 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.311080+0000", 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1143000000000006e-05 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:2078 3.2 3:40fd9093:::rbd_data.10f16a447f06.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.3338655030000002, 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044032740000000004, 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2078 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411814+0000", 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.018e-06 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411820+0000", 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919999999999997e-06 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411838+0000", 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7943000000000001e-05 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411846+0000", 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9529999999999999e-06 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411973+0000", 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012721400000000001 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412087+0000", 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011305499999999999 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412223+0000", 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136427 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.413226+0000", 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001002635 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416125+0000", 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028995459999999998 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416176+0000", 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1347000000000002e-05 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416216+0000", 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9243999999999999e-05 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:2084 3.0 3:07e4adab:::rbd_data.10f16a447f06.0000000000000063:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.412046+0000", 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.3336321600000005, 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036908219999999999, 2024-09-21T23:15:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2084 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412046+0000", 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412046+0000", 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412047+0000", 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.403e-06 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412060+0000", 2024-09-21T23:15:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2405000000000001e-05 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412075+0000", 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5267000000000001e-05 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412085+0000", 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9990000000000003e-06 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412196+0000", 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110899 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412326+0000", 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013063300000000001 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412498+0000", 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017202999999999999 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.413802+0000", 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001303329 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.415612+0000", 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00181025 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.415684+0000", 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1981000000000001e-05 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.415737+0000", 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2626000000000001e-05 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:5981 3.0 3:03ba3e89:::rbd_data.10f1e443a6fa.00000000000001b3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.445544+0000", 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.30013421899999998, 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026875559999999998, 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 5981 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445544+0000", 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445544+0000", 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445546+0000", 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-06 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445553+0000", 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3499999999999999e-06 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445576+0000", 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2467999999999999e-05 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445589+0000", 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3028e-05 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445683+0000", 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4349000000000001e-05 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445756+0000", 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2389000000000001e-05 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445852+0000", 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6218999999999995e-05 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.447035+0000", 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001183108 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.448171+0000", 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011365660000000001 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.448208+0000", 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6078000000000003e-05 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.448231+0000", 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3740999999999999e-05 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:5984 3.0 3:03dda2cf:::rbd_data.10f1e443a6fa.00000000000001b7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.445637+0000", 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.300040637, 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003397424, 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 5984 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445637+0000", 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445637+0000", 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445638+0000", 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6200000000000006e-07 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445643+0000", 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1460000000000004e-06 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445668+0000", 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4542999999999999e-05 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.445679+0000", 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1384e-05 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.446313+0000", 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000633283 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.446501+0000", 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000188409 2024-09-21T23:15:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.446638+0000", 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013734700000000001 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.447546+0000", 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090771000000000005 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.448982+0000", 2024-09-21T23:15:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001435399 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.449014+0000", 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2032000000000001e-05 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.449035+0000", 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209e-05 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6062 3.0 3:063318eb:::rbd_data.10f1e443a6fa.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.485293+0000", 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.26038540199999999, 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028072650000000002, 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6062 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485293+0000", 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485293+0000", 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485294+0000", 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.866e-06 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485301+0000", 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8179999999999998e-06 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485321+0000", 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9544000000000001e-05 2024-09-21T23:15:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485332+0000", 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1253999999999999e-05 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485413+0000", 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0663999999999999e-05 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485529+0000", 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000116041 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.485604+0000", 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5599000000000006e-05 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.486223+0000", 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061909499999999995 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.487906+0000", 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001682883 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.488045+0000", 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013826499999999999 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.488100+0000", 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5235999999999998e-05 2024-09-21T23:15:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.25352888499999998, 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6396 3.2 3:454c9511:::rbd_data.10f1e443a6fa.0000000000000043:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619096+0000", 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.12658156700000001, 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033722159999999999, 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6396 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619096+0000", 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619096+0000", 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619097+0000", 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4200000000000005e-07 2024-09-21T23:15:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619107+0000", 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3070000000000008e-06 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619119+0000", 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2797e-05 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619128+0000", 2024-09-21T23:15:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5739999999999996e-06 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619241+0000", 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000113423 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619405+0000", 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016327099999999999 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619591+0000", 2024-09-21T23:15:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018619099999999999 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620899+0000", 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013079420000000001 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.622401+0000", 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015023440000000001 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.622431+0000", 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9538000000000001e-05 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.622469+0000", 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7987000000000001e-05 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.12631187299999999, 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:16.854 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:16.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:15:16.855 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:16.847+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:16.856 INFO:teuthology.orchestra.run.smithi170.stdout:423 write 0xe9e2600 thru 0xe9e53ce (0x2dcf bytes) 2024-09-21T23:15:16.856 INFO:teuthology.orchestra.run.smithi170.stdout:426 read 0x847c400 thru 0x8486a85 (0xa686 bytes) 2024-09-21T23:15:16.856 INFO:teuthology.orchestra.run.smithi170.stdout:429 trunc from 0xe9e5400 to 0x50a3600 2024-09-21T23:15:16.856 INFO:teuthology.orchestra.run.smithi170.stdout:430 write 0x2a36000 thru 0x2a3d6fc (0x76fd bytes) 2024-09-21T23:15:16.856 INFO:teuthology.orchestra.run.smithi170.stdout:432 trunc from 0x50a3600 to 0x799a000 2024-09-21T23:15:16.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:16.862 INFO:teuthology.orchestra.run.smithi170.stdout:433 read 0x63a5600 thru 0x63ad993 (0x8394 bytes) 2024-09-21T23:15:16.863 INFO:teuthology.orchestra.run.smithi170.stdout:434 punch from 0x29e7a00 to 0x29ec600, (0x4c00 bytes) 2024-09-21T23:15:16.863 INFO:teuthology.orchestra.run.smithi170.stdout:435 write 0x7802400 thru 0x7808738 (0x6339 bytes) 2024-09-21T23:15:16.865 INFO:teuthology.orchestra.run.smithi170.stdout:438 write 0x539b000 thru 0x53a52a1 (0xa2a2 bytes) 2024-09-21T23:15:16.865 INFO:teuthology.orchestra.run.smithi170.stdout:440 punch from 0x6994000 to 0x6998e00, (0x4e00 bytes) 2024-09-21T23:15:16.866 INFO:teuthology.orchestra.run.smithi170.stdout:443 read 0x2128600 thru 0x21372c5 (0xecc6 bytes) 2024-09-21T23:15:16.866 INFO:teuthology.orchestra.run.smithi170.stdout:444 read 0x48b3200 thru 0x48bc4b3 (0x92b4 bytes) 2024-09-21T23:15:16.869 INFO:teuthology.orchestra.run.smithi170.stdout:445 trunc from 0x799a000 to 0x8e44c00 2024-09-21T23:15:16.922 INFO:teuthology.orchestra.run.smithi170.stdout:446 write 0x909f000 thru 0x90a6cf6 (0x7cf7 bytes) 2024-09-21T23:15:16.924 INFO:teuthology.orchestra.run.smithi170.stdout:448 write 0x806ac00 thru 0x807473f (0x9b40 bytes) 2024-09-21T23:15:16.926 INFO:teuthology.orchestra.run.smithi170.stdout:449 trunc from 0x90a6e00 to 0x8226a00 2024-09-21T23:15:16.956 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:16.951+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:16.997 INFO:teuthology.orchestra.run.smithi170.stdout:450 write 0xe22cc00 thru 0xe230b8f (0x3f90 bytes) 2024-09-21T23:15:17.000 INFO:teuthology.orchestra.run.smithi170.stdout:454 write 0x259f400 thru 0x25a26f5 (0x32f6 bytes) 2024-09-21T23:15:17.000 INFO:teuthology.orchestra.run.smithi170.stdout:455 read 0x2ad6e00 thru 0x2ae6dc0 (0xffc1 bytes) 2024-09-21T23:15:17.006 INFO:teuthology.orchestra.run.smithi170.stdout:458 trunc from 0xe230c00 to 0xc53b400 2024-09-21T23:15:17.035 INFO:teuthology.orchestra.run.smithi170.stdout:460 read 0x8e0a800 thru 0x8e10104 (0x5905 bytes) 2024-09-21T23:15:17.037 INFO:teuthology.orchestra.run.smithi170.stdout:462 read 0x1788400 thru 0x1797ad0 (0xf6d1 bytes) 2024-09-21T23:15:17.047 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:17.047 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:17.047 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:17.047 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:17.056 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:17.056+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:17.059 INFO:teuthology.orchestra.run.smithi170.stdout:463 trunc from 0xc53b400 to 0x110e200 2024-09-21T23:15:17.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:17.117 INFO:teuthology.orchestra.run.smithi170.stdout:465 trunc from 0x110e200 to 0x90df400 2024-09-21T23:15:17.129 INFO:teuthology.orchestra.run.smithi170.stdout:466 punch from 0x44f6800 to 0x4500600, (0x9e00 bytes) 2024-09-21T23:15:17.131 INFO:teuthology.orchestra.run.smithi170.stdout:469 read 0x3197a00 thru 0x319d5b5 (0x5bb6 bytes) 2024-09-21T23:15:17.131 INFO:teuthology.orchestra.run.smithi170.stdout:470 read 0x81b3a00 thru 0x81b3fa5 (0x5a6 bytes) 2024-09-21T23:15:17.131 INFO:teuthology.orchestra.run.smithi170.stdout:471 punch from 0x1a20c00 to 0x1a2ec00, (0xe000 bytes) 2024-09-21T23:15:17.131 INFO:teuthology.orchestra.run.smithi170.stdout:472 read 0x64f4600 thru 0x65014e3 (0xcee4 bytes) 2024-09-21T23:15:17.137 INFO:teuthology.orchestra.run.smithi170.stdout:476 trunc from 0x90df400 to 0x59d5000 2024-09-21T23:15:17.157 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:17.151+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:17.165 INFO:teuthology.orchestra.run.smithi170.stdout:479 punch from 0xbee600 to 0xbfd400, (0xee00 bytes) 2024-09-21T23:15:17.167 INFO:teuthology.orchestra.run.smithi170.stdout:480 read 0x122ba00 thru 0x123ad99 (0xf39a bytes) 2024-09-21T23:15:17.167 INFO:teuthology.orchestra.run.smithi170.stdout:482 read 0x4937400 thru 0x4938b80 (0x1781 bytes) 2024-09-21T23:15:17.170 INFO:teuthology.orchestra.run.smithi170.stdout:485 trunc from 0x59d5000 to 0x48f0000 2024-09-21T23:15:17.218 INFO:teuthology.orchestra.run.smithi170.stdout:487 write 0xdc44c00 thru 0xdc4a12a (0x552b bytes) 2024-09-21T23:15:17.243 INFO:teuthology.orchestra.run.smithi170.stdout:489 write 0xe98b200 thru 0xe997e01 (0xcc02 bytes) 2024-09-21T23:15:17.244 INFO:teuthology.orchestra.run.smithi170.stdout:491 read 0x774c200 thru 0x77578f4 (0xb6f5 bytes) 2024-09-21T23:15:17.245 INFO:teuthology.orchestra.run.smithi170.stdout:492 read 0x7d3a600 thru 0x7d42c02 (0x8603 bytes) 2024-09-21T23:15:17.245 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:17.245 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:17.256 INFO:teuthology.orchestra.run.smithi170.stdout:494 trunc from 0xe998000 to 0xa580a00 2024-09-21T23:15:17.257 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:17.256+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:17.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:17.289 INFO:teuthology.orchestra.run.smithi170.stdout:497 read 0xa1ca800 thru 0xa1cc7b0 (0x1fb1 bytes) 2024-09-21T23:15:17.291 INFO:teuthology.orchestra.run.smithi170.stdout:498 punch from 0x5ca400 to 0x5d5200, (0xae00 bytes) 2024-09-21T23:15:17.314 INFO:teuthology.orchestra.run.smithi170.stdout:500 trunc from 0xa580a00 to 0x16f2400 2024-09-21T23:15:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: marking collection for removal 2024-09-21T23:15:17.317 INFO:teuthology.orchestra.run.smithi053.stdout:setting '_remove' omap key 2024-09-21T23:15:17.317 INFO:teuthology.orchestra.run.smithi053.stdout:finish_remove_pgs 3.10_head removing 3.10 2024-09-21T23:15:17.318 INFO:teuthology.orchestra.run.smithi053.stdout:Remove successful 2024-09-21T23:15:17.338 INFO:teuthology.orchestra.run.smithi170.stdout:503 read 0x886200 thru 0x88c3f4 (0x61f5 bytes) 2024-09-21T23:15:17.341 INFO:teuthology.orchestra.run.smithi170.stdout:504 read 0x3cee00 thru 0x3d6f08 (0x8109 bytes) 2024-09-21T23:15:17.342 INFO:teuthology.orchestra.run.smithi170.stdout:506 read 0x1143c00 thru 0x1146c8a (0x308b bytes) 2024-09-21T23:15:17.348 INFO:teuthology.orchestra.run.smithi170.stdout:508 trunc from 0x16f2400 to 0x3a40c00 2024-09-21T23:15:17.358 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:17.356+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:17.365 INFO:teuthology.orchestra.run.smithi170.stdout:509 trunc from 0x3a40c00 to 0x759a400 2024-09-21T23:15:17.377 INFO:teuthology.orchestra.run.smithi170.stdout:512 read 0x68bf000 thru 0x68cd136 (0xe137 bytes) 2024-09-21T23:15:17.379 INFO:teuthology.orchestra.run.smithi170.stdout:514 read 0x46fb000 thru 0x4701750 (0x6751 bytes) 2024-09-21T23:15:17.379 INFO:teuthology.orchestra.run.smithi170.stdout:515 write 0x46b6400 thru 0x46c4093 (0xdc94 bytes) 2024-09-21T23:15:17.379 INFO:teuthology.orchestra.run.smithi170.stdout:517 read 0xd82c00 thru 0xd90ec7 (0xe2c8 bytes) 2024-09-21T23:15:17.406 INFO:teuthology.orchestra.run.smithi170.stdout:518 write 0x89d7600 thru 0x89d7e80 (0x881 bytes) 2024-09-21T23:15:17.408 INFO:teuthology.orchestra.run.smithi170.stdout:519 punch from 0x238e00 to 0x241400, (0x8600 bytes) 2024-09-21T23:15:17.408 INFO:teuthology.orchestra.run.smithi170.stdout:520 read 0x6b95a00 thru 0x6ba2d4b (0xd34c bytes) 2024-09-21T23:15:17.409 INFO:teuthology.orchestra.run.smithi170.stdout:521 read 0xd14000 thru 0xd1db64 (0x9b65 bytes) 2024-09-21T23:15:17.412 INFO:teuthology.orchestra.run.smithi170.stdout:522 write 0x46ce400 thru 0x46d2545 (0x4146 bytes) 2024-09-21T23:15:17.423 INFO:teuthology.orchestra.run.smithi170.stdout:523 trunc from 0x89d8000 to 0xc509a00 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:4918 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.026421+0000", 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.4152997979999999, 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012740708999999999, 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4918 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.026421+0000", 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.026421+0000", 2024-09-21T23:15:17.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.026421+0000", 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999998e-07 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.026424+0000", 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1070000000000001e-06 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.026434+0000", 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0006999999999999e-05 2024-09-21T23:15:17.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.026445+0000", 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.037400+0000", 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010955266999999999 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.037504+0000", 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010419799999999999 2024-09-21T23:15:17.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.037949+0000", 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044505400000000001 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.038625+0000", 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067613700000000005 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.039038+0000", 2024-09-21T23:15:17.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000412996 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.039104+0000", 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5951000000000006e-05 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.039162+0000", 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7139999999999998e-05 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:5920 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.422969+0000", 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018751441, 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014381992, 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 5920 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.422969+0000", 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.422969+0000", 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.422970+0000", 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.422972+0000", 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6129999999999998e-06 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.422977+0000", 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1900000000000003e-06 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.422981+0000", 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5039999999999998e-06 2024-09-21T23:15:17.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.432735+0000", 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097546030000000006 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.432973+0000", 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000237605 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.433686+0000", 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071263900000000005 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.437237+0000", 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035511039999999998 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.437257+0000", 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0429999999999999e-05 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.437305+0000", 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7880000000000002e-05 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.437351+0000", 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6053000000000001e-05 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6561 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.708943+0000", 2024-09-21T23:15:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.73277809000000005, 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012865829000000001, 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6561 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708943+0000", 2024-09-21T23:15:17.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708943+0000", 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708943+0000", 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8599999999999999e-07 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708945+0000", 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5050000000000002e-06 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708950+0000", 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7559999999999999e-06 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708954+0000", 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4829999999999999e-06 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.717269+0000", 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0083153140000000007 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.717439+0000", 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000170225 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.717992+0000", 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055324599999999997 2024-09-21T23:15:17.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.721578+0000", 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035854070000000001 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.721673+0000", 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5276000000000001e-05 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.721749+0000", 2024-09-21T23:15:17.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6087999999999994e-05 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.721808+0000", 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9243000000000002e-05 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6562 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.708958+0000", 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.73276250700000001, 2024-09-21T23:15:17.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013076177, 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6562 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708958+0000", 2024-09-21T23:15:17.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708958+0000", 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708958+0000", 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8099999999999999e-07 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708961+0000", 2024-09-21T23:15:17.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2790000000000001e-06 2024-09-21T23:15:17.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708965+0000", 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6229999999999996e-06 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708969+0000", 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6550000000000002e-06 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.719189+0000", 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010220119999999999 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.719313+0000", 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124401 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.719726+0000", 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041293900000000003 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.721879+0000", 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002152321 2024-09-21T23:15:17.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.721890+0000", 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1355999999999999e-05 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.721966+0000", 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6358000000000003e-05 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.722034+0000", 2024-09-21T23:15:17.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7843999999999996e-05 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6563 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.708976+0000", 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.73274466699999996, 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013421860000000001, 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.466 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6563 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708976+0000", 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708976+0000", 2024-09-21T23:15:17.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708976+0000", 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0499999999999999e-07 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708979+0000", 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.379e-06 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708984+0000", 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9439999999999996e-06 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.708987+0000", 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.72e-06 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.720366+0000", 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011378218000000001 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.720479+0000", 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000113464 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.720865+0000", 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038568899999999999 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.722049+0000", 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011838759999999999 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.722289+0000", 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024024300000000001 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.722357+0000", 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7756000000000007e-05 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.722398+0000", 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1266000000000001e-05 2024-09-21T23:15:17.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6998 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.867616+0000", 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57410504799999995, 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012821078, 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.472 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6998 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867616+0000", 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867616+0000", 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867617+0000", 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3489999999999999e-06 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867626+0000", 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0939999999999997e-06 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867661+0000", 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4666000000000003e-05 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867679+0000", 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8105e-05 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.875885+0000", 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0082067170000000005 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.876087+0000", 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000201852 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.876611+0000", 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052351800000000005 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.880316+0000", 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037046959999999999 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.880337+0000", 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1121999999999999e-05 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.880394+0000", 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6847e-05 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.880437+0000", 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3112000000000003e-05 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6999 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.867748+0000", 2024-09-21T23:15:17.477 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57397308000000002, 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014053304000000001, 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6999 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867748+0000", 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867748+0000", 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867749+0000", 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.088e-06 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867757+0000", 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2549999999999997e-06 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867774+0000", 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6877000000000001e-05 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867783+0000", 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9709999999999993e-06 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.877783+0000", 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010000491 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.877875+0000", 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1623e-05 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.878276+0000", 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040157700000000002 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.880517+0000", 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022407830000000001 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.881690+0000", 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011729189999999999 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.881752+0000", 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1811999999999997e-05 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.881801+0000", 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8908e-05 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.484 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7000 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.867826+0000", 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57389509100000002, 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015294241, 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7000 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867826+0000", 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867826+0000", 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867827+0000", 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1030000000000001e-06 2024-09-21T23:15:17.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867837+0000", 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9429999999999995e-06 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867863+0000", 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6886999999999999e-05 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867873+0000", 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2e-06 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.879083+0000", 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011210606 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.879246+0000", 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000162547 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.879698+0000", 2024-09-21T23:15:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045197800000000002 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.882955+0000", 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032575569999999999 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.882977+0000", 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1644000000000001e-05 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.883058+0000", 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380999999999999e-05 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.883120+0000", 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1395000000000004e-05 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7001 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.490 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.867886+0000", 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57383478200000004, 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016745665999999999, 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7001 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867886+0000", 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867886+0000", 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867887+0000", 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0299999999999998e-07 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867891+0000", 2024-09-21T23:15:17.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-06 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867902+0000", 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.047e-05 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867910+0000", 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9969999999999998e-06 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.880591+0000", 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012681133000000001 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.880679+0000", 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7707000000000006e-05 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.881099+0000", 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042062099999999998 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.883188+0000", 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020888500000000002 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.884482+0000", 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012943550000000001 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.884550+0000", 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7501000000000003e-05 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.884631+0000", 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1649000000000004e-05 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7002 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.867929+0000", 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57379140200000001, 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018084571000000001, 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7002 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867929+0000", 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867929+0000", 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867930+0000", 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7000000000000003e-07 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867937+0000", 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867958+0000", 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.867978+0000", 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0126999999999999e-05 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.881885+0000", 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013907553 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.881980+0000", 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4251000000000003e-05 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.882397+0000", 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041704200000000001 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.885901+0000", 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003504849 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.885913+0000", 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1438e-05 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.885970+0000", 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6867000000000003e-05 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.886014+0000", 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4051000000000002e-05 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7004 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.868008+0000", 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57371236999999997, 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019179753000000001, 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.502 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7004 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868008+0000", 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868008+0000", 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868010+0000", 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.547e-06 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868037+0000", 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7656000000000001e-05 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868058+0000", 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0886999999999998e-05 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868079+0000", 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0193000000000001e-05 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.883260+0000", 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015181435 2024-09-21T23:15:17.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.883373+0000", 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011294999999999999 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.883841+0000", 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000467918 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.886128+0000", 2024-09-21T23:15:17.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022867180000000001 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.887073+0000", 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094576600000000001 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.887139+0000", 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5760999999999996e-05 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.887188+0000", 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8921999999999997e-05 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7006 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.868215+0000", 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57350526400000001, 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020500994000000002, 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.508 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7006 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868215+0000", 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868215+0000", 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868217+0000", 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5030000000000001e-06 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868226+0000", 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8829999999999995e-06 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868242+0000", 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6520000000000001e-05 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868252+0000", 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8970000000000003e-06 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.884741+0000", 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016488714000000002 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.884847+0000", 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105845 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.885312+0000", 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046555700000000002 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888478+0000", 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031656589999999999 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888505+0000", 2024-09-21T23:15:17.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7234e-05 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888625+0000", 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000119985 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888716+0000", 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1197000000000001e-05 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7007 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.868268+0000", 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.57345280099999996, 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020734051, 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7007 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868268+0000", 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868268+0000", 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868269+0000", 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8000000000000005e-07 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868273+0000", 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4830000000000001e-06 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868283+0000", 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8959999999999998e-06 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868291+0000", 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6299999999999998e-06 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.886166+0000", 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017874944 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.886230+0000", 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4008000000000001e-05 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.886560+0000", 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033026900000000001 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888747+0000", 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021873109999999999 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888903+0000", 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015545500000000001 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888946+0000", 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3220000000000003e-05 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.889002+0000", 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6054999999999998e-05 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7008 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.868301+0000", 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.573420029, 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022368945000000001, 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7008 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868301+0000", 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868301+0000", 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868301+0000", 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3e-07 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868305+0000", 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0470000000000004e-06 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868317+0000", 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1367e-05 2024-09-21T23:15:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.868324+0000", 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9800000000000001e-06 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.887240+0000", 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018915988000000002 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.887341+0000", 2024-09-21T23:15:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010143 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.887762+0000", 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042121699999999999 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.888758+0000", 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099613799999999993 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.890547+0000", 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788347 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.890618+0000", 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1631000000000006e-05 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.890670+0000", 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.117e-05 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7168 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.929555+0000", 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.51216516099999998, 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01334573, 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7168 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929555+0000", 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929555+0000", 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929556+0000", 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-07 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929558+0000", 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9439999999999999e-06 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929562+0000", 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7319999999999998e-06 2024-09-21T23:15:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929565+0000", 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8430000000000001e-06 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.940293+0000", 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010727429 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.940385+0000", 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2189999999999997e-05 2024-09-21T23:15:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.940779+0000", 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039366500000000002 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.942701+0000", 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001922776 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.942721+0000", 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9740000000000001e-05 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.942784+0000", 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2589999999999995e-05 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.942901+0000", 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117514 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7169 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.929569+0000", 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.512151248, 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014109079, 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7169 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929569+0000", 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929569+0000", 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929570+0000", 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.28e-07 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929571+0000", 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6419999999999999e-06 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929575+0000", 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5549999999999998e-06 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.929578+0000", 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.886e-06 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.941745+0000", 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012167802 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.941835+0000", 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9357999999999994e-05 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.942199+0000", 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000364261 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.943029+0000", 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083029299999999996 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.943518+0000", 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048855899999999996 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.943606+0000", 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7603999999999997e-05 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.943678+0000", 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2891000000000004e-05 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7734 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.133823+0000", 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.307897592, 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012764795000000001, 2024-09-21T23:15:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7734 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133823+0000", 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133823+0000", 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133823+0000", 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8599999999999998e-07 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133827+0000", 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9260000000000002e-06 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133838+0000", 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.022e-05 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133844+0000", 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3679999999999998e-06 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.144326+0000", 2024-09-21T23:15:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010482183000000001 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.144414+0000", 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7498999999999997e-05 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.144790+0000", 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037584800000000002 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.146331+0000", 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015416309999999999 2024-09-21T23:15:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.146527+0000", 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195696 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.146552+0000", 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4888999999999998e-05 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.146588+0000", 2024-09-21T23:15:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6049e-05 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7735 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.133852+0000", 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.30786821399999997, 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015147620000000001, 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7735 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133852+0000", 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133852+0000", 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133853+0000", 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133856+0000", 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1310000000000002e-06 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133865+0000", 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.448e-06 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133871+0000", 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8590000000000001e-06 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.145342+0000", 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011471067 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.145427+0000", 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5142999999999994e-05 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.145776+0000", 2024-09-21T23:15:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034880399999999998 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.148817+0000", 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030402290000000002 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.148862+0000", 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.148958+0000", 2024-09-21T23:15:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5913000000000001e-05 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149000+0000", 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1696000000000002e-05 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7736 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.133879+0000", 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.307841692, 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015277394999999999, 2024-09-21T23:15:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7736 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133879+0000", 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133879+0000", 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133879+0000", 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133882+0000", 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133891+0000", 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7439999999999996e-06 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133897+0000", 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.879e-06 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.146630+0000", 2024-09-21T23:15:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01273294 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.146715+0000", 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5554000000000001e-05 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.147083+0000", 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000367924 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.148825+0000", 2024-09-21T23:15:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017414360000000001 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149088+0000", 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026297000000000001 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149130+0000", 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2097e-05 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149156+0000", 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6410999999999998e-05 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:7737 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.133905+0000", 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.307816015, 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015886931, 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7737 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133905+0000", 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133905+0000", 2024-09-21T23:15:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133905+0000", 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133908+0000", 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9179999999999998e-06 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133915+0000", 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2370000000000003e-06 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.133921+0000", 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.147786+0000", 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013865519999999999 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.147884+0000", 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7868000000000006e-05 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.148347+0000", 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000463311 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149171+0000", 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082422099999999998 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149705+0000", 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053342600000000004 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149754+0000", 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9595999999999997e-05 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.149792+0000", 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7070999999999998e-05 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:17.559 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:17.561 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:524 write 0xcd80c00 thru 0xcd877a3 (0x6ba4 bytes) 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:525 read 0x5e2fc00 thru 0x5e3775e (0x7b5f bytes) 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:528 write 0xb3c1000 thru 0xb3c57ce (0x47cf bytes) 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:532 punch from 0x1394800 to 0x139ba00, (0x7200 bytes) 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:533 punch from 0xb3bd600 to 0xb3c2c00, (0x5600 bytes) 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:535 trunc from 0xcd87800 to 0xa82ec00 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:536 write 0xe711800 thru 0xe7207a1 (0xefa2 bytes) 2024-09-21T23:15:17.562 INFO:teuthology.orchestra.run.smithi170.stdout:538 punch from 0x6d9c800 to 0x6d9ee00, (0x2600 bytes) 2024-09-21T23:15:17.563 INFO:teuthology.orchestra.run.smithi170.stdout:542 read 0x606fe00 thru 0x6071775 (0x1976 bytes) 2024-09-21T23:15:17.563 INFO:teuthology.orchestra.run.smithi170.stdout:543 read 0x1d50e00 thru 0x1d5fd71 (0xef72 bytes) 2024-09-21T23:15:17.563 INFO:teuthology.orchestra.run.smithi170.stdout:545 punch from 0x6dca600 to 0x6dcae00, (0x800 bytes) 2024-09-21T23:15:17.563 INFO:teuthology.orchestra.run.smithi170.stdout:546 write 0x15f800 thru 0x16f03c (0xf83d bytes) 2024-09-21T23:15:17.563 INFO:teuthology.orchestra.run.smithi170.stdout:547 read 0xcc72e00 thru 0xcc7e1fe (0xb3ff bytes) 2024-09-21T23:15:17.563 INFO:teuthology.orchestra.run.smithi170.stdout:549 write 0x99db000 thru 0x99e8f12 (0xdf13 bytes) 2024-09-21T23:15:17.563 INFO:teuthology.orchestra.run.smithi170.stdout:550 trunc from 0xe720800 to 0xa409200 2024-09-21T23:15:17.563 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:17.555+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:17.581 INFO:teuthology.orchestra.run.smithi170.stdout:551 write 0x24d9600 thru 0x24e8e76 (0xf877 bytes) 2024-09-21T23:15:17.616 INFO:teuthology.orchestra.run.smithi170.stdout:552 write 0xc17fe00 thru 0xc18b482 (0xb683 bytes) 2024-09-21T23:15:17.618 INFO:teuthology.orchestra.run.smithi170.stdout:553 read 0x6401c00 thru 0x6404c1c (0x301d bytes) 2024-09-21T23:15:17.624 INFO:teuthology.orchestra.run.smithi170.stdout:555 trunc from 0xc18b600 to 0x95c3a00 2024-09-21T23:15:17.661 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:17.660+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:17.683 INFO:teuthology.orchestra.run.smithi170.stdout:559 write 0x9a7e800 thru 0x9a8052c (0x1d2d bytes) 2024-09-21T23:15:17.705 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:17.706 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:17.706 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:17.706 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:17.710 INFO:teuthology.orchestra.run.smithi170.stdout:560 write 0xabeb200 thru 0xabf95d9 (0xe3da bytes) 2024-09-21T23:15:17.711 INFO:teuthology.orchestra.run.smithi170.stdout:561 write 0x6794400 thru 0x679f907 (0xb508 bytes) 2024-09-21T23:15:17.711 INFO:teuthology.orchestra.run.smithi170.stdout:562 punch from 0x7576e00 to 0x757e000, (0x7200 bytes) 2024-09-21T23:15:17.717 INFO:teuthology.orchestra.run.smithi170.stdout:563 trunc from 0xabf9600 to 0x907de00 2024-09-21T23:15:17.720 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:17.767 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:17.763+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:17.771 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:15:17.776 INFO:teuthology.orchestra.run.smithi170.stdout:565 trunc from 0x907de00 to 0xdde3e00 2024-09-21T23:15:17.791 INFO:teuthology.orchestra.run.smithi170.stdout:566 punch from 0x4df5400 to 0x4e01000, (0xbc00 bytes) 2024-09-21T23:15:17.793 INFO:teuthology.orchestra.run.smithi170.stdout:567 write 0x8ad9800 thru 0x8ae6dc4 (0xd5c5 bytes) 2024-09-21T23:15:17.794 INFO:teuthology.orchestra.run.smithi170.stdout:568 read 0x9170e00 thru 0x917a69c (0x989d bytes) 2024-09-21T23:15:17.794 INFO:teuthology.orchestra.run.smithi170.stdout:569 punch from 0xd10400 to 0xd18c00, (0x8800 bytes) 2024-09-21T23:15:17.795 INFO:teuthology.orchestra.run.smithi170.stdout:571 write 0x6a08400 thru 0x6a0bc0c (0x380d bytes) 2024-09-21T23:15:17.797 INFO:teuthology.orchestra.run.smithi170.stdout:572 read 0x8ea2e00 thru 0x8eacbc3 (0x9dc4 bytes) 2024-09-21T23:15:17.797 INFO:teuthology.orchestra.run.smithi170.stdout:573 punch from 0x7aeaa00 to 0x7af4600, (0x9c00 bytes) 2024-09-21T23:15:17.798 INFO:teuthology.orchestra.run.smithi170.stdout:575 read 0x6f4fc00 thru 0x6f58193 (0x8594 bytes) 2024-09-21T23:15:17.815 INFO:teuthology.orchestra.run.smithi170.stdout:579 trunc from 0xdde3e00 to 0x5095c00 2024-09-21T23:15:17.855 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:17.855 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:17.855 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:17.855 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:17.855 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:17.863 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:17.860+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:17.865 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.10.0 2024-09-21T23:15:17.869 INFO:teuthology.orchestra.run.smithi170.stdout:581 write 0x75a9600 thru 0x75b0ed9 (0x78da bytes) 2024-09-21T23:15:17.869 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:17.871 INFO:teuthology.orchestra.run.smithi170.stdout:583 read 0xce1800 thru 0xce6009 (0x480a bytes) 2024-09-21T23:15:17.872 INFO:teuthology.orchestra.run.smithi170.stdout:584 punch from 0x52d5e00 to 0x52d7800, (0x1a00 bytes) 2024-09-21T23:15:17.880 INFO:teuthology.orchestra.run.smithi170.stdout:586 trunc from 0x75b1000 to 0x3619600 2024-09-21T23:15:17.912 INFO:teuthology.orchestra.run.smithi170.stdout:588 write 0x9bc000 thru 0x9cad8a (0xed8b bytes) 2024-09-21T23:15:17.914 INFO:teuthology.orchestra.run.smithi170.stdout:589 punch from 0x2e53c00 to 0x2e5a600, (0x6a00 bytes) 2024-09-21T23:15:17.932 INFO:teuthology.orchestra.run.smithi170.stdout:590 write 0x6631400 thru 0x6634158 (0x2d59 bytes) 2024-09-21T23:15:17.938 INFO:teuthology.orchestra.run.smithi170.stdout:591 trunc from 0x6634200 to 0x89ff200 2024-09-21T23:15:17.956 INFO:teuthology.orchestra.run.smithi170.stdout:593 punch from 0x66f1400 to 0x66f4800, (0x3400 bytes) 2024-09-21T23:15:17.958 INFO:teuthology.orchestra.run.smithi170.stdout:594 punch from 0x5ead800 to 0x5eb9e00, (0xc600 bytes) 2024-09-21T23:15:17.958 INFO:teuthology.orchestra.run.smithi170.stdout:595 read 0x7189600 thru 0x718e85a (0x525b bytes) 2024-09-21T23:15:17.959 INFO:teuthology.orchestra.run.smithi170.stdout:598 write 0x3a98200 thru 0x3a9b340 (0x3141 bytes) 2024-09-21T23:15:17.959 INFO:teuthology.orchestra.run.smithi170.stdout:600 write 0x605f200 thru 0x606b10d (0xbf0e bytes) 2024-09-21T23:15:17.959 INFO:teuthology.orchestra.run.smithi170.stdout:602 write 0x3dec600 thru 0x3df213f (0x5b40 bytes) 2024-09-21T23:15:17.964 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:17.959+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:17.967 INFO:teuthology.orchestra.run.smithi170.stdout:606 trunc from 0x89ff200 to 0x3a14000 2024-09-21T23:15:18.020 INFO:teuthology.orchestra.run.smithi170.stdout:608 trunc from 0x3a14000 to 0xe1e6800 2024-09-21T23:15:18.034 INFO:teuthology.orchestra.run.smithi170.stdout:609 read 0x1994400 thru 0x199b122 (0x6d23 bytes) 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.819206898000001, 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.044 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.050 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.306801615000001, 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:18.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:18.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.306736711999999, 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.056 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:18.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.306590273000001, 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.306521157999999, 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.305908833, 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.305704427999999, 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.305661642, 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914703354, 2024-09-21T23:15:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914616844000001, 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914539954999999, 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914335115, 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914246833, 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914116005, 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914070352, 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.914031925, 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.913902462999999, 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.913880820999999, 2024-09-21T23:15:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.913860777, 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 30.913776785, 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:18.157 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:18.157 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:18.158 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:18.156+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:18.159 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:610 trunc from 0xe1e6800 to 0x7cae800 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:611 read 0x1665c00 thru 0x16738ed (0xdcee bytes) 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:613 write 0x3d23c00 thru 0x3d2cb55 (0x8f56 bytes) 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:616 read 0x21fac00 thru 0x220554e (0xa94f bytes) 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:618 read 0x2c5d400 thru 0x2c6408e (0x6c8f bytes) 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:619 write 0x3f23800 thru 0x3f24df8 (0x15f9 bytes) 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:620 write 0x883c200 thru 0x8844db6 (0x8bb7 bytes) 2024-09-21T23:15:18.160 INFO:teuthology.orchestra.run.smithi170.stdout:622 punch from 0x3260600 to 0x326e200, (0xdc00 bytes) 2024-09-21T23:15:18.161 INFO:teuthology.orchestra.run.smithi170.stdout:624 write 0x1bf1c00 thru 0x1bfb9f9 (0x9dfa bytes) 2024-09-21T23:15:18.161 INFO:teuthology.orchestra.run.smithi170.stdout:625 trunc from 0x8844e00 to 0x16d4e00 2024-09-21T23:15:18.227 INFO:teuthology.orchestra.run.smithi170.stdout:627 write 0x3468000 thru 0x3468a18 (0xa19 bytes) 2024-09-21T23:15:18.257 INFO:teuthology.orchestra.run.smithi170.stdout:628 write 0xc358200 thru 0xc35bee4 (0x3ce5 bytes) 2024-09-21T23:15:18.258 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:18.256+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:18.259 INFO:teuthology.orchestra.run.smithi170.stdout:629 write 0x1217e00 thru 0x1218681 (0x882 bytes) 2024-09-21T23:15:18.259 INFO:teuthology.orchestra.run.smithi170.stdout:630 read 0x8c59800 thru 0x8c652d9 (0xbada bytes) 2024-09-21T23:15:18.259 INFO:teuthology.orchestra.run.smithi170.stdout:631 punch from 0x4ee1600 to 0x4ee2800, (0x1200 bytes) 2024-09-21T23:15:18.260 INFO:teuthology.orchestra.run.smithi170.stdout:632 read 0x79bec00 thru 0x79cb115 (0xc516 bytes) 2024-09-21T23:15:18.260 INFO:teuthology.orchestra.run.smithi170.stdout:633 read 0xb409400 thru 0xb4124fd (0x90fe bytes) 2024-09-21T23:15:18.262 INFO:teuthology.orchestra.run.smithi170.stdout:636 trunc from 0xc35c000 to 0xdae7000 2024-09-21T23:15:18.282 INFO:teuthology.orchestra.run.smithi170.stdout:637 write 0x56c8800 thru 0x56d02ca (0x7acb bytes) 2024-09-21T23:15:18.283 INFO:teuthology.orchestra.run.smithi170.stdout:639 write 0xd24800 thru 0xd32b02 (0xe303 bytes) 2024-09-21T23:15:18.284 INFO:teuthology.orchestra.run.smithi170.stdout:642 punch from 0x9dda800 to 0x9de8800, (0xe000 bytes) 2024-09-21T23:15:18.284 INFO:teuthology.orchestra.run.smithi170.stdout:644 read 0x16a3400 thru 0x16a6ac2 (0x36c3 bytes) 2024-09-21T23:15:18.286 INFO:teuthology.orchestra.run.smithi170.stdout:645 write 0x7bab400 thru 0x7bad065 (0x1c66 bytes) 2024-09-21T23:15:18.286 INFO:teuthology.orchestra.run.smithi170.stdout:646 read 0x51f1a00 thru 0x51f8697 (0x6c98 bytes) 2024-09-21T23:15:18.287 INFO:teuthology.orchestra.run.smithi170.stdout:648 read 0xa67b800 thru 0xa684dbb (0x95bc bytes) 2024-09-21T23:15:18.287 INFO:teuthology.orchestra.run.smithi170.stdout:649 punch from 0x7d0d400 to 0x7d1a400, (0xd000 bytes) 2024-09-21T23:15:18.287 INFO:teuthology.orchestra.run.smithi170.stdout:652 punch from 0x80ea600 to 0x80f2c00, (0x8600 bytes) 2024-09-21T23:15:18.287 INFO:teuthology.orchestra.run.smithi170.stdout:654 write 0x48fbc00 thru 0x49017c2 (0x5bc3 bytes) 2024-09-21T23:15:18.294 INFO:teuthology.orchestra.run.smithi170.stdout:655 trunc from 0xdae7000 to 0xa40a800 2024-09-21T23:15:18.310 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:18.334 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:18.334 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:18.337 INFO:teuthology.orchestra.run.smithi170.stdout:656 read 0x56c9c00 thru 0x56cc0d7 (0x24d8 bytes) 2024-09-21T23:15:18.340 INFO:teuthology.orchestra.run.smithi170.stdout:658 write 0x9f20e00 thru 0x9f2f942 (0xeb43 bytes) 2024-09-21T23:15:18.341 INFO:teuthology.orchestra.run.smithi170.stdout:660 write 0xa118200 thru 0xa11e994 (0x6795 bytes) 2024-09-21T23:15:18.349 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:18.349 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:18.349 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:18.349 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:18.352 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:18.352 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:18.360 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:18.355+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:18.365 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:18.371 INFO:teuthology.orchestra.run.smithi170.stdout:663 write 0xeb01400 thru 0xeb0b9a7 (0xa5a8 bytes) 2024-09-21T23:15:18.372 INFO:teuthology.orchestra.run.smithi170.stdout:664 write 0x1cb4800 thru 0x1cc4249 (0xfa4a bytes) 2024-09-21T23:15:18.372 INFO:teuthology.orchestra.run.smithi170.stdout:665 write 0xbd17600 thru 0xbd1e185 (0x6b86 bytes) 2024-09-21T23:15:18.372 INFO:teuthology.orchestra.run.smithi170.stdout:668 read 0x68d5e00 thru 0x68dd01a (0x721b bytes) 2024-09-21T23:15:18.373 INFO:teuthology.orchestra.run.smithi170.stdout:672 punch from 0x4adc800 to 0x4ae1800, (0x5000 bytes) 2024-09-21T23:15:18.374 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:18.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:18.403 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:18.403 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:18.404 INFO:teuthology.orchestra.run.smithi170.stdout:674 trunc from 0xeb0ba00 to 0xd8aa00 2024-09-21T23:15:18.432 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:18.432 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:18.435 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:18.435 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:18.461 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:18.455+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:18.482 INFO:teuthology.orchestra.run.smithi170.stdout:675 trunc from 0xd8aa00 to 0xa6a3200 2024-09-21T23:15:18.501 INFO:teuthology.orchestra.run.smithi170.stdout:676 trunc from 0xa6a3200 to 0xd021800 2024-09-21T23:15:18.514 INFO:teuthology.orchestra.run.smithi170.stdout:677 write 0x49c9800 thru 0x49cf438 (0x5c39 bytes) 2024-09-21T23:15:18.516 INFO:teuthology.orchestra.run.smithi170.stdout:678 write 0xa82a000 thru 0xa82c731 (0x2732 bytes) 2024-09-21T23:15:18.516 INFO:teuthology.orchestra.run.smithi170.stdout:679 write 0x5262800 thru 0x526d2fd (0xaafe bytes) 2024-09-21T23:15:18.516 INFO:teuthology.orchestra.run.smithi170.stdout:680 punch from 0xa30c400 to 0xa313800, (0x7400 bytes) 2024-09-21T23:15:18.517 INFO:teuthology.orchestra.run.smithi170.stdout:682 read 0x388000 thru 0x396c5d (0xec5e bytes) 2024-09-21T23:15:18.519 INFO:teuthology.orchestra.run.smithi170.stdout:683 read 0x89ffa00 thru 0x8a0c724 (0xcd25 bytes) 2024-09-21T23:15:18.519 INFO:teuthology.orchestra.run.smithi170.stdout:684 write 0x711a200 thru 0x7124bae (0xa9af bytes) 2024-09-21T23:15:18.520 INFO:teuthology.orchestra.run.smithi170.stdout:685 write 0x4587000 thru 0x459691a (0xf91b bytes) 2024-09-21T23:15:18.520 INFO:teuthology.orchestra.run.smithi170.stdout:686 punch from 0x5251800 to 0x5254200, (0x2a00 bytes) 2024-09-21T23:15:18.522 INFO:teuthology.orchestra.run.smithi170.stdout:689 read 0x8168a00 thru 0x8169644 (0xc45 bytes) 2024-09-21T23:15:18.522 INFO:teuthology.orchestra.run.smithi170.stdout:690 write 0x3b20e00 thru 0x3b303c0 (0xf5c1 bytes) 2024-09-21T23:15:18.522 INFO:teuthology.orchestra.run.smithi170.stdout:691 read 0x77ca600 thru 0x77d4503 (0x9f04 bytes) 2024-09-21T23:15:18.522 INFO:teuthology.orchestra.run.smithi170.stdout:693 punch from 0x7160200 to 0x7166e00, (0x6c00 bytes) 2024-09-21T23:15:18.544 INFO:teuthology.orchestra.run.smithi170.stdout:694 trunc from 0xd021800 to 0xf1bc00 2024-09-21T23:15:18.546 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:18.546 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:18.561 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:18.555+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:18.563 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:18.613 INFO:teuthology.orchestra.run.smithi170.stdout:696 read 0xaae400 thru 0xab171e (0x331f bytes) 2024-09-21T23:15:18.638 INFO:teuthology.orchestra.run.smithi170.stdout:699 write 0x5d45000 thru 0x5d47777 (0x2778 bytes) 2024-09-21T23:15:18.639 INFO:teuthology.orchestra.run.smithi170.stdout:701 read 0x5b66600 thru 0x5b670c5 (0xac6 bytes) 2024-09-21T23:15:18.639 INFO:teuthology.orchestra.run.smithi170.stdout:702 punch from 0x2e0d600 to 0x2e19200, (0xbc00 bytes) 2024-09-21T23:15:18.642 INFO:teuthology.orchestra.run.smithi170.stdout:703 read 0x1bf1800 thru 0x1bfaae9 (0x92ea bytes) 2024-09-21T23:15:18.642 INFO:teuthology.orchestra.run.smithi170.stdout:704 read 0x45f0600 thru 0x45f550b (0x4f0c bytes) 2024-09-21T23:15:18.642 INFO:teuthology.orchestra.run.smithi170.stdout:707 read 0x2852a00 thru 0x285c7b1 (0x9db2 bytes) 2024-09-21T23:15:18.642 INFO:teuthology.orchestra.run.smithi170.stdout:710 read 0x1149600 thru 0x11520cf (0x8ad0 bytes) 2024-09-21T23:15:18.642 INFO:teuthology.orchestra.run.smithi170.stdout:712 write 0x145b200 thru 0x1460558 (0x5359 bytes) 2024-09-21T23:15:18.642 INFO:teuthology.orchestra.run.smithi170.stdout:713 trunc from 0x5d47800 to 0x6422200 2024-09-21T23:15:18.662 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:18.660+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:18.693 INFO:teuthology.orchestra.run.smithi170.stdout:721 write 0x6e10a00 thru 0x6e11320 (0x921 bytes) 2024-09-21T23:15:18.694 INFO:teuthology.orchestra.run.smithi170.stdout:722 punch from 0x3382c00 to 0x3389200, (0x6600 bytes) 2024-09-21T23:15:18.694 INFO:teuthology.orchestra.run.smithi170.stdout:723 read 0x2e0d200 thru 0x2e1b848 (0xe649 bytes) 2024-09-21T23:15:18.714 INFO:teuthology.orchestra.run.smithi170.stdout:727 trunc from 0x6e11400 to 0x263fe00 2024-09-21T23:15:18.736 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:18.736 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:18.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:18.763 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:18.760+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:18.777 INFO:teuthology.orchestra.run.smithi170.stdout:728 write 0x991c600 thru 0x99210f3 (0x4af4 bytes) 2024-09-21T23:15:18.787 INFO:teuthology.orchestra.run.smithi170.stdout:730 trunc from 0x9921200 to 0xe170800 2024-09-21T23:15:18.815 INFO:teuthology.orchestra.run.smithi170.stdout:731 write 0x43b5400 thru 0x43c1a42 (0xc643 bytes) 2024-09-21T23:15:18.816 INFO:teuthology.orchestra.run.smithi170.stdout:734 read 0x1231a00 thru 0x123fb59 (0xe15a bytes) 2024-09-21T23:15:18.832 INFO:teuthology.orchestra.run.smithi170.stdout:735 trunc from 0xe170800 to 0x87a4400 2024-09-21T23:15:18.864 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:18.860+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:18.875 INFO:teuthology.orchestra.run.smithi170.stdout:736 trunc from 0x87a4400 to 0x81cea00 2024-09-21T23:15:18.876 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:18.876 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:18.891 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:18.891 INFO:teuthology.orchestra.run.smithi170.stdout:737 punch from 0x6a35c00 to 0x6a36200, (0x600 bytes) 2024-09-21T23:15:18.920 INFO:teuthology.orchestra.run.smithi170.stdout:738 write 0xebdda00 thru 0xebddb1a (0x11b bytes) 2024-09-21T23:15:18.922 INFO:teuthology.orchestra.run.smithi170.stdout:739 read 0x9d32800 thru 0x9d350e3 (0x28e4 bytes) 2024-09-21T23:15:18.923 INFO:teuthology.orchestra.run.smithi170.stdout:742 punch from 0xc727e00 to 0xc731000, (0x9200 bytes) 2024-09-21T23:15:18.923 INFO:teuthology.orchestra.run.smithi170.stdout:748 punch from 0xde95000 to 0xde9f200, (0xa200 bytes) 2024-09-21T23:15:18.923 INFO:teuthology.orchestra.run.smithi170.stdout:749 read 0xe22d000 thru 0xe232b81 (0x5b82 bytes) 2024-09-21T23:15:18.924 INFO:teuthology.orchestra.run.smithi170.stdout:751 read 0xc913e00 thru 0xc91c2c8 (0x84c9 bytes) 2024-09-21T23:15:18.924 INFO:teuthology.orchestra.run.smithi170.stdout:752 read 0xe778400 thru 0xe783580 (0xb181 bytes) 2024-09-21T23:15:18.924 INFO:teuthology.orchestra.run.smithi170.stdout:756 punch from 0xd5b8c00 to 0xd5c4a00, (0xbe00 bytes) 2024-09-21T23:15:18.924 INFO:teuthology.orchestra.run.smithi170.stdout:truncating to largest ever: 0xeac8000 2024-09-21T23:15:18.924 INFO:teuthology.orchestra.run.smithi170.stdout:759 trunc from 0xebddc00 to 0xeac8000 2024-09-21T23:15:18.959 INFO:teuthology.orchestra.run.smithi170.stdout:761 write 0x256c200 thru 0x25795f7 (0xd3f8 bytes) 2024-09-21T23:15:18.961 INFO:teuthology.orchestra.run.smithi170.stdout:762 write 0x63da800 thru 0x63e08ba (0x60bb bytes) 2024-09-21T23:15:18.965 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:18.959+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:18.991 INFO:teuthology.orchestra.run.smithi170.stdout:767 trunc from 0xeac8000 to 0x1ded600 2024-09-21T23:15:19.026 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:19.027 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:19.042 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:19.066 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:19.064+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:19.104 INFO:teuthology.orchestra.run.smithi170.stdout:768 punch from 0x1457e00 to 0x1460a00, (0x8c00 bytes) 2024-09-21T23:15:19.107 INFO:teuthology.orchestra.run.smithi170.stdout:771 write 0x1945c00 thru 0x1951b54 (0xbf55 bytes) 2024-09-21T23:15:19.166 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:19.159+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:19.171 INFO:teuthology.orchestra.run.smithi170.stdout:774 write 0xc293800 thru 0xc29ae32 (0x7633 bytes) 2024-09-21T23:15:19.172 INFO:teuthology.orchestra.run.smithi170.stdout:778 punch from 0x3f38c00 to 0x3f3b400, (0x2800 bytes) 2024-09-21T23:15:19.173 INFO:teuthology.orchestra.run.smithi170.stdout:779 write 0x2c75c00 thru 0x2c76f62 (0x1363 bytes) 2024-09-21T23:15:19.181 INFO:teuthology.orchestra.run.smithi170.stdout:780 trunc from 0xc29b000 to 0x7c8f000 2024-09-21T23:15:19.216 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:19.216 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:19.216 INFO:teuthology.orchestra.run.smithi170.stdout:782 read 0x755f000 thru 0x756b686 (0xc687 bytes) 2024-09-21T23:15:19.217 INFO:teuthology.orchestra.run.smithi170.stdout:784 read 0x7b7f200 thru 0x7b81e7b (0x2c7c bytes) 2024-09-21T23:15:19.217 INFO:teuthology.orchestra.run.smithi170.stdout:789 read 0x3381200 thru 0x338713c (0x5f3d bytes) 2024-09-21T23:15:19.217 INFO:teuthology.orchestra.run.smithi170.stdout:791 write 0x5f2a400 thru 0x5f32e64 (0x8a65 bytes) 2024-09-21T23:15:19.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:19.232 INFO:teuthology.orchestra.run.smithi170.stdout:792 trunc from 0x7c8f000 to 0xd04f200 2024-09-21T23:15:19.259 INFO:teuthology.orchestra.run.smithi170.stdout:795 read 0x9a72200 thru 0x9a753fd (0x31fe bytes) 2024-09-21T23:15:19.262 INFO:teuthology.orchestra.run.smithi170.stdout:797 write 0x6405c00 thru 0x640621a (0x61b bytes) 2024-09-21T23:15:19.262 INFO:teuthology.orchestra.run.smithi170.stdout:798 write 0x2ff800 thru 0x30b334 (0xbb35 bytes) 2024-09-21T23:15:19.263 INFO:teuthology.orchestra.run.smithi170.stdout:799 punch from 0xc083800 to 0xc085200, (0x1a00 bytes) 2024-09-21T23:15:19.264 INFO:teuthology.orchestra.run.smithi170.stdout:800 write 0xa192200 thru 0xa19a401 (0x8202 bytes) 2024-09-21T23:15:19.264 INFO:teuthology.orchestra.run.smithi170.stdout:804 write 0x5ea9800 thru 0x5eb3840 (0xa041 bytes) 2024-09-21T23:15:19.265 INFO:teuthology.orchestra.run.smithi170.stdout:805 read 0x4ef7c00 thru 0x4f02715 (0xab16 bytes) 2024-09-21T23:15:19.275 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:19.271+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:19.275 INFO:teuthology.orchestra.run.smithi170.stdout:806 trunc from 0xd04f200 to 0x7fa2000 2024-09-21T23:15:19.314 INFO:teuthology.orchestra.run.smithi170.stdout:808 trunc from 0x7fa2000 to 0x764e600 2024-09-21T23:15:19.337 INFO:teuthology.orchestra.run.smithi170.stdout:810 trunc from 0x764e600 to 0x4661a00 2024-09-21T23:15:19.362 INFO:teuthology.orchestra.run.smithi170.stdout:811 read 0x1920000 thru 0x192b19b (0xb19c bytes) 2024-09-21T23:15:19.368 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:19.363+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:19.370 INFO:teuthology.orchestra.run.smithi170.stdout:812 trunc from 0x4661a00 to 0x5bfe800 2024-09-21T23:15:19.391 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:19.391 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:19.395 INFO:teuthology.orchestra.run.smithi170.stdout:814 trunc from 0x5bfe800 to 0x1b5f800 2024-09-21T23:15:19.409 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:19.437 INFO:teuthology.orchestra.run.smithi170.stdout:816 write 0x6c37200 thru 0x6c40cb9 (0x9aba bytes) 2024-09-21T23:15:19.439 INFO:teuthology.orchestra.run.smithi170.stdout:819 write 0x55d2200 thru 0x55d5813 (0x3614 bytes) 2024-09-21T23:15:19.439 INFO:teuthology.orchestra.run.smithi170.stdout:820 read 0x976c00 thru 0x976cc7 (0xc8 bytes) 2024-09-21T23:15:19.439 INFO:teuthology.orchestra.run.smithi170.stdout:823 read 0x662ca00 thru 0x6633008 (0x6609 bytes) 2024-09-21T23:15:19.442 INFO:teuthology.orchestra.run.smithi170.stdout:824 trunc from 0x6c40e00 to 0x55e2400 2024-09-21T23:15:19.469 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:19.468+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:19.481 INFO:teuthology.orchestra.run.smithi170.stdout:825 read 0x2990200 thru 0x2997ad0 (0x78d1 bytes) 2024-09-21T23:15:19.484 INFO:teuthology.orchestra.run.smithi170.stdout:828 read 0x3044200 thru 0x3053ad6 (0xf8d7 bytes) 2024-09-21T23:15:19.484 INFO:teuthology.orchestra.run.smithi170.stdout:830 write 0x3502200 thru 0x3510ad6 (0xe8d7 bytes) 2024-09-21T23:15:19.485 INFO:teuthology.orchestra.run.smithi170.stdout:831 punch from 0x54f0c00 to 0x54fd000, (0xc400 bytes) 2024-09-21T23:15:19.486 INFO:teuthology.orchestra.run.smithi170.stdout:833 read 0xd90c00 thru 0xd95fb0 (0x53b1 bytes) 2024-09-21T23:15:19.486 INFO:teuthology.orchestra.run.smithi170.stdout:834 read 0xc52600 thru 0xc52be7 (0x5e8 bytes) 2024-09-21T23:15:19.487 INFO:teuthology.orchestra.run.smithi170.stdout:835 write 0x3846e00 thru 0x3848779 (0x197a bytes) 2024-09-21T23:15:19.487 INFO:teuthology.orchestra.run.smithi170.stdout:839 read 0x2c90600 thru 0x2c97893 (0x7294 bytes) 2024-09-21T23:15:19.487 INFO:teuthology.orchestra.run.smithi170.stdout:842 write 0x2b61e00 thru 0x2b703c0 (0xe5c1 bytes) 2024-09-21T23:15:19.487 INFO:teuthology.orchestra.run.smithi170.stdout:843 read 0x3cc2200 thru 0x3cc2e7f (0xc80 bytes) 2024-09-21T23:15:19.487 INFO:teuthology.orchestra.run.smithi170.stdout:844 write 0x2bb800 thru 0x2c8241 (0xca42 bytes) 2024-09-21T23:15:19.488 INFO:teuthology.orchestra.run.smithi170.stdout:846 punch from 0x1501600 to 0x1507200, (0x5c00 bytes) 2024-09-21T23:15:19.490 INFO:teuthology.orchestra.run.smithi170.stdout:847 trunc from 0x55e2400 to 0x4cb8800 2024-09-21T23:15:19.545 INFO:teuthology.orchestra.run.smithi170.stdout:849 read 0x18e1200 thru 0x18e3112 (0x1f13 bytes) 2024-09-21T23:15:19.549 INFO:teuthology.orchestra.run.smithi170.stdout:850 write 0x91d600 thru 0x927fb8 (0xa9b9 bytes) 2024-09-21T23:15:19.551 INFO:teuthology.orchestra.run.smithi170.stdout:851 read 0x486200 thru 0x4940e3 (0xdee4 bytes) 2024-09-21T23:15:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:19.555 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:19.570 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:19.563+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:19.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:19.579 INFO:teuthology.orchestra.run.smithi170.stdout:852 write 0x80f5e00 thru 0x81026e8 (0xc8e9 bytes) 2024-09-21T23:15:19.580 INFO:teuthology.orchestra.run.smithi170.stdout:855 read 0x2283a00 thru 0x2284fdd (0x15de bytes) 2024-09-21T23:15:19.580 INFO:teuthology.orchestra.run.smithi170.stdout:857 punch from 0x49d5400 to 0x49e3400, (0xe000 bytes) 2024-09-21T23:15:19.582 INFO:teuthology.orchestra.run.smithi170.stdout:860 write 0xdaa800 thru 0xdba6ae (0xfeaf bytes) 2024-09-21T23:15:19.582 INFO:teuthology.orchestra.run.smithi170.stdout:862 read 0x6221600 thru 0x622bc76 (0xa677 bytes) 2024-09-21T23:15:19.582 INFO:teuthology.orchestra.run.smithi170.stdout:863 read 0x134f200 thru 0x13591ca (0x9fcb bytes) 2024-09-21T23:15:19.601 INFO:teuthology.orchestra.run.smithi170.stdout:871 read 0x683ee00 thru 0x684b650 (0xc851 bytes) 2024-09-21T23:15:19.640 INFO:teuthology.orchestra.run.smithi170.stdout:872 write 0xd4f7e00 thru 0xd4ff469 (0x766a bytes) 2024-09-21T23:15:19.643 INFO:teuthology.orchestra.run.smithi170.stdout:873 read 0x8c51800 thru 0x8c56e5e (0x565f bytes) 2024-09-21T23:15:19.643 INFO:teuthology.orchestra.run.smithi170.stdout:875 write 0x1bdcc00 thru 0x1be42f4 (0x76f5 bytes) 2024-09-21T23:15:19.643 INFO:teuthology.orchestra.run.smithi170.stdout:876 write 0x78c1000 thru 0x78cd25d (0xc25e bytes) 2024-09-21T23:15:19.644 INFO:teuthology.orchestra.run.smithi170.stdout:877 write 0x3968e00 thru 0x3970dba (0x7fbb bytes) 2024-09-21T23:15:19.644 INFO:teuthology.orchestra.run.smithi170.stdout:879 punch from 0x3570600 to 0x357e600, (0xe000 bytes) 2024-09-21T23:15:19.646 INFO:teuthology.orchestra.run.smithi170.stdout:883 write 0x3eaac00 thru 0x3eb685f (0xbc60 bytes) 2024-09-21T23:15:19.646 INFO:teuthology.orchestra.run.smithi170.stdout:884 read 0x764ac00 thru 0x7650bb2 (0x5fb3 bytes) 2024-09-21T23:15:19.651 INFO:teuthology.orchestra.run.smithi170.stdout:886 trunc from 0xd4ff600 to 0xb36f400 2024-09-21T23:15:19.671 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:19.668+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:19.704 INFO:teuthology.orchestra.run.smithi170.stdout:888 punch from 0x35ef800 to 0x35fa600, (0xae00 bytes) 2024-09-21T23:15:19.706 INFO:teuthology.orchestra.run.smithi170.stdout:889 read 0xb2fe00 thru 0xb3827f (0x8480 bytes) 2024-09-21T23:15:19.710 INFO:teuthology.orchestra.run.smithi170.stdout:892 read 0x288e00 thru 0x28c0cb (0x32cc bytes) 2024-09-21T23:15:19.714 INFO:teuthology.orchestra.run.smithi170.stdout:895 write 0x8b4c200 thru 0x8b4e21a (0x201b bytes) 2024-09-21T23:15:19.714 INFO:teuthology.orchestra.run.smithi170.stdout:897 write 0x8749e00 thru 0x8758a87 (0xec88 bytes) 2024-09-21T23:15:19.715 INFO:teuthology.orchestra.run.smithi170.stdout:898 write 0x37c800 thru 0x37f8a1 (0x30a2 bytes) 2024-09-21T23:15:19.715 INFO:teuthology.orchestra.run.smithi170.stdout:899 punch from 0x5001c00 to 0x5009000, (0x7400 bytes) 2024-09-21T23:15:19.716 INFO:teuthology.orchestra.run.smithi170.stdout:901 read 0x35ee400 thru 0x35fbd41 (0xd942 bytes) 2024-09-21T23:15:19.728 INFO:teuthology.orchestra.run.smithi170.stdout:903 trunc from 0xb36f400 to 0x716c200 2024-09-21T23:15:19.762 INFO:teuthology.orchestra.run.smithi170.stdout:904 write 0x1da7200 thru 0x1db0aa8 (0x98a9 bytes) 2024-09-21T23:15:19.762 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:19.763 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:19.765 INFO:teuthology.orchestra.run.smithi170.stdout:906 read 0x3a44a00 thru 0x3a52790 (0xdd91 bytes) 2024-09-21T23:15:19.767 INFO:teuthology.orchestra.run.smithi170.stdout:908 punch from 0x20e1a00 to 0x20ee000, (0xc600 bytes) 2024-09-21T23:15:19.768 INFO:teuthology.orchestra.run.smithi170.stdout:910 punch from 0x215f400 to 0x216a800, (0xb400 bytes) 2024-09-21T23:15:19.771 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:19.767+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:19.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:19.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:15:19.801 INFO:teuthology.orchestra.run.smithi170.stdout:911 write 0x9d14c00 thru 0x9d172c7 (0x26c8 bytes) 2024-09-21T23:15:19.803 INFO:teuthology.orchestra.run.smithi170.stdout:913 read 0x7880400 thru 0x78810f1 (0xcf2 bytes) 2024-09-21T23:15:19.803 INFO:teuthology.orchestra.run.smithi170.stdout:914 write 0x9c7ea00 thru 0x9c8df4c (0xf54d bytes) 2024-09-21T23:15:19.803 INFO:teuthology.orchestra.run.smithi170.stdout:916 read 0x60db400 thru 0x60e96a4 (0xe2a5 bytes) 2024-09-21T23:15:19.803 INFO:teuthology.orchestra.run.smithi170.stdout:920 punch from 0x2b73200 to 0x2b82800, (0xf600 bytes) 2024-09-21T23:15:19.804 INFO:teuthology.orchestra.run.smithi170.stdout:921 punch from 0x9ac6400 to 0x9ad1e00, (0xba00 bytes) 2024-09-21T23:15:19.804 INFO:teuthology.orchestra.run.smithi170.stdout:922 punch from 0x389ac00 to 0x38a8200, (0xd600 bytes) 2024-09-21T23:15:19.805 INFO:teuthology.orchestra.run.smithi170.stdout:923 read 0x6319c00 thru 0x631a11e (0x51f bytes) 2024-09-21T23:15:19.805 INFO:teuthology.orchestra.run.smithi170.stdout:929 write 0x4909600 thru 0x4915906 (0xc307 bytes) 2024-09-21T23:15:19.835 INFO:teuthology.orchestra.run.smithi170.stdout:930 write 0xbfe6000 thru 0xbfe814d (0x214e bytes) 2024-09-21T23:15:19.836 INFO:teuthology.orchestra.run.smithi170.stdout:931 read 0x938e00 thru 0x941527 (0x8728 bytes) 2024-09-21T23:15:19.838 INFO:teuthology.orchestra.run.smithi170.stdout:933 write 0x48d9600 thru 0x48da870 (0x1271 bytes) 2024-09-21T23:15:19.840 INFO:teuthology.orchestra.run.smithi170.stdout:936 read 0x5699200 thru 0x56a4c6d (0xba6e bytes) 2024-09-21T23:15:19.840 INFO:teuthology.orchestra.run.smithi170.stdout:937 punch from 0xa988000 to 0xa990600, (0x8600 bytes) 2024-09-21T23:15:19.841 INFO:teuthology.orchestra.run.smithi170.stdout:939 read 0xb66fc00 thru 0xb67ec10 (0xf011 bytes) 2024-09-21T23:15:19.841 INFO:teuthology.orchestra.run.smithi170.stdout:941 punch from 0x6c0ba00 to 0x6c19c00, (0xe200 bytes) 2024-09-21T23:15:19.842 INFO:teuthology.orchestra.run.smithi170.stdout:942 write 0xc67200 thru 0xc6f2f7 (0x80f8 bytes) 2024-09-21T23:15:19.860 INFO:teuthology.orchestra.run.smithi170.stdout:944 trunc from 0xbfe8200 to 0x5a93800 2024-09-21T23:15:19.872 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:19.872+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:19.896 INFO:teuthology.orchestra.run.smithi170.stdout:945 read 0x2a1800 thru 0x2aca32 (0xb233 bytes) 2024-09-21T23:15:19.910 INFO:teuthology.orchestra.run.smithi170.stdout:946 trunc from 0x5a93800 to 0x1e39800 2024-09-21T23:15:19.961 INFO:teuthology.orchestra.run.smithi170.stdout:947 punch from 0x168ce00 to 0x1699400, (0xc600 bytes) 2024-09-21T23:15:19.963 INFO:teuthology.orchestra.run.smithi170.stdout:950 read 0x1295200 thru 0x129cc78 (0x7a79 bytes) 2024-09-21T23:15:19.965 INFO:teuthology.orchestra.run.smithi170.stdout:952 write 0x553600 thru 0x560293 (0xcc94 bytes) 2024-09-21T23:15:19.968 INFO:teuthology.orchestra.run.smithi170.stdout:953 punch from 0x34ee00 to 0x359a00, (0xac00 bytes) 2024-09-21T23:15:19.969 INFO:teuthology.orchestra.run.smithi170.stdout:956 read 0x1102200 thru 0x110aa55 (0x8856 bytes) 2024-09-21T23:15:19.972 INFO:teuthology.orchestra.run.smithi170.stdout:957 read 0xe9fe00 thru 0xea78b9 (0x7aba bytes) 2024-09-21T23:15:19.973 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:19.972+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:19.974 INFO:teuthology.orchestra.run.smithi170.stdout:958 punch from 0x1396800 to 0x1398a00, (0x2200 bytes) 2024-09-21T23:15:19.982 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:19.983 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:19.983 INFO:teuthology.orchestra.run.smithi170.stdout:959 trunc from 0x1e39800 to 0x4c80e00 2024-09-21T23:15:19.993 INFO:teuthology.orchestra.run.smithi053.stdout:get_pg_num_history pg_num_history pg_num_history(e29 pg_nums {1={13=1},2={16=8},3={21=32}} deleted_pools ) 2024-09-21T23:15:19.994 INFO:teuthology.orchestra.run.smithi053.stdout:Importing pgid 3.10 2024-09-21T23:15:19.994 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:0eb5383f:::rbd_data.10f16a447f06.0000000000000005:head# 2024-09-21T23:15:19.994 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-21T23:15:19.998 INFO:teuthology.orchestra.run.smithi053.stdout:write_pg epoch 28 info 3.10( v 28'15 (0'0,28'15] local-lis/les=21/22 n=1 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-21T23:15:19.998 INFO:teuthology.orchestra.run.smithi053.stdout:Import successful 2024-09-21T23:15:20.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:20.019 INFO:teuthology.orchestra.run.smithi170.stdout:961 write 0x70aa600 thru 0x70b6a25 (0xc426 bytes) 2024-09-21T23:15:20.031 INFO:teuthology.orchestra.run.smithi170.stdout:962 trunc from 0x70b6c00 to 0xb65da00 2024-09-21T23:15:20.054 INFO:teuthology.orchestra.run.smithi170.stdout:965 write 0x94fe800 thru 0x950a752 (0xbf53 bytes) 2024-09-21T23:15:20.088 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:20.088+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:20.094 INFO:teuthology.orchestra.run.smithi170.stdout:966 write 0xed44c00 thru 0xed507d3 (0xbbd4 bytes) 2024-09-21T23:15:20.096 INFO:teuthology.orchestra.run.smithi170.stdout:969 read 0xd1fe400 thru 0xd20cc98 (0xe899 bytes) 2024-09-21T23:15:20.096 INFO:teuthology.orchestra.run.smithi170.stdout:970 read 0x30cfe00 thru 0x30dcf40 (0xd141 bytes) 2024-09-21T23:15:20.097 INFO:teuthology.orchestra.run.smithi170.stdout:971 read 0xd16a00 thru 0xd1ec02 (0x8203 bytes) 2024-09-21T23:15:20.098 INFO:teuthology.orchestra.run.smithi170.stdout:976 write 0x596aa00 thru 0x5972deb (0x83ec bytes) 2024-09-21T23:15:20.099 INFO:teuthology.orchestra.run.smithi170.stdout:977 read 0x6f3c00 thru 0x6fbed7 (0x82d8 bytes) 2024-09-21T23:15:20.100 INFO:teuthology.orchestra.run.smithi170.stdout:980 write 0xc48ca00 thru 0xc49c9e0 (0xffe1 bytes) 2024-09-21T23:15:20.100 INFO:teuthology.orchestra.run.smithi170.stdout:981 punch from 0x64cf600 to 0x64dac00, (0xb600 bytes) 2024-09-21T23:15:20.122 INFO:teuthology.orchestra.run.smithi170.stdout:982 trunc from 0xed50800 to 0x416d000 2024-09-21T23:15:20.175 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:20.167+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:20.195 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:20.195 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:20.211 INFO:teuthology.orchestra.run.smithi170.stdout:985 write 0x9017e00 thru 0x901f4cb (0x76cc bytes) 2024-09-21T23:15:20.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:986 read 0x7402200 thru 0x740a26f (0x8070 bytes) 2024-09-21T23:15:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:987 read 0x3b6d000 thru 0x3b77c16 (0xac17 bytes) 2024-09-21T23:15:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:988 read 0x48c0000 thru 0x48c8673 (0x8674 bytes) 2024-09-21T23:15:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:990 read 0x8084e00 thru 0x808cdfc (0x7ffd bytes) 2024-09-21T23:15:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:994 punch from 0x5170a00 to 0x5179200, (0x8800 bytes) 2024-09-21T23:15:20.214 INFO:teuthology.orchestra.run.smithi170.stdout:996 read 0x6ad4c00 thru 0x6ad50ec (0x4ed bytes) 2024-09-21T23:15:20.224 INFO:teuthology.orchestra.run.smithi170.stdout:999 trunc from 0x901f600 to 0xc9c2000 2024-09-21T23:15:20.263 INFO:teuthology.orchestra.run.smithi170.stdout:1002 trunc from 0xc9c2000 to 0x2c1f000 2024-09-21T23:15:20.290 INFO:teuthology.orchestra.run.smithi170.stdout:1003 trunc from 0x2c1f000 to 0x219400 2024-09-21T23:15:20.300 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:20.300+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:20.332 INFO:teuthology.orchestra.run.smithi170.stdout:1006 punch from 0x1a7e00 to 0x1ac600, (0x4800 bytes) 2024-09-21T23:15:20.352 INFO:teuthology.orchestra.run.smithi170.stdout:1011 trunc from 0x219400 to 0x6f1c600 2024-09-21T23:15:20.376 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:20.376+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:20.389 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:20.389 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:20.395 INFO:teuthology.orchestra.run.smithi170.stdout:1012 write 0xdd71200 thru 0xdd8004f (0xee50 bytes) 2024-09-21T23:15:20.398 INFO:teuthology.orchestra.run.smithi170.stdout:1014 punch from 0xad3a000 to 0xad48200, (0xe200 bytes) 2024-09-21T23:15:20.399 INFO:teuthology.orchestra.run.smithi170.stdout:1015 punch from 0x5b9ca00 to 0x5baa800, (0xde00 bytes) 2024-09-21T23:15:20.399 INFO:teuthology.orchestra.run.smithi170.stdout:1019 punch from 0x3c31e00 to 0x3c3ae00, (0x9000 bytes) 2024-09-21T23:15:20.399 INFO:teuthology.orchestra.run.smithi170.stdout:1020 read 0x8659400 thru 0x8661844 (0x8445 bytes) 2024-09-21T23:15:20.403 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:20.415 INFO:teuthology.orchestra.run.smithi170.stdout:1022 trunc from 0xdd80200 to 0x3d7b000 2024-09-21T23:15:20.437 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:20.496 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:20.496+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:20.510 INFO:teuthology.orchestra.run.smithi170.stdout:1023 write 0xbaf0000 thru 0xbaf5dcc (0x5dcd bytes) 2024-09-21T23:15:20.511 INFO:teuthology.orchestra.run.smithi170.stdout:1026 read 0x8a62000 thru 0x8a663c3 (0x43c4 bytes) 2024-09-21T23:15:20.514 INFO:teuthology.orchestra.run.smithi170.stdout:1027 trunc from 0xbaf5e00 to 0xc7f9600 2024-09-21T23:15:20.530 INFO:teuthology.orchestra.run.smithi170.stdout:1029 read 0x30e7e00 thru 0x30e900a (0x120b bytes) 2024-09-21T23:15:20.531 INFO:teuthology.orchestra.run.smithi170.stdout:1030 punch from 0x2b4c400 to 0x2b50200, (0x3e00 bytes) 2024-09-21T23:15:20.531 INFO:teuthology.orchestra.run.smithi170.stdout:1031 write 0x9e6c000 thru 0x9e782f6 (0xc2f7 bytes) 2024-09-21T23:15:20.546 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:20.546 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:20.547 INFO:teuthology.orchestra.run.smithi170.stdout:1033 trunc from 0xc7f9600 to 0x6cdf400 2024-09-21T23:15:20.559 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /var/log/ceph/exp.3.10.0 2024-09-21T23:15:20.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:20.579 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:20.580 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:20.576+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:20.580 INFO:teuthology.orchestra.run.smithi170.stdout:1035 read 0x10af000 thru 0x10b78b0 (0x88b1 bytes) 2024-09-21T23:15:20.597 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:20.597 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:20.613 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:20.613 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:20.630 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:20.630 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:20.647 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:20.667 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:20.667 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:20.670 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:20.671 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:20.679 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:20.671+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:20.682 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:20.682 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:20.682 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:20.682 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:20.683 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:20.696 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:20.783 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:20.779+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:20.792 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:15:20.795 INFO:teuthology.orchestra.run.smithi170.stdout:1036 punch from 0x5371a00 to 0x5377c00, (0x6200 bytes) 2024-09-21T23:15:20.798 INFO:teuthology.orchestra.run.smithi170.stdout:1037 trunc from 0x6cdf400 to 0x5ae6000 2024-09-21T23:15:20.829 INFO:teuthology.orchestra.run.smithi170.stdout:1038 trunc from 0x5ae6000 to 0x189f600 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.647631425999997, 2024-09-21T23:15:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:20.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:20.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:20.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.135226142999997, 2024-09-21T23:15:20.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:20.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.135161240000002, 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.885 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:20.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:20.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.135014800999997, 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:20.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.896 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.134945686000002, 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:20.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:20.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.902 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.134333361000003, 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.903 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.134128955999998, 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.909 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:20.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:20.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:20.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.134086170000003, 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:20.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:20.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.743127882000003, 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:20.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.743041372, 2024-09-21T23:15:20.929 INFO:teuthology.orchestra.run.smithi170.stdout:1041 read 0x158e200 thru 0x1597dc8 (0x9bc9 bytes) 2024-09-21T23:15:20.929 INFO:teuthology.orchestra.run.smithi170.stdout:1042 read 0x620600 thru 0x6301d7 (0xfbd8 bytes) 2024-09-21T23:15:20.929 INFO:teuthology.orchestra.run.smithi170.stdout:1043 trunc from 0x189f600 to 0xad3ba00 2024-09-21T23:15:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742964483000002, 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742759642999999, 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742671360999999, 2024-09-21T23:15:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742540533000003, 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742494880000002, 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742456453000003, 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742326990999999, 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:20.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:20.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742305348999999, 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:20.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:20.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742285305000003, 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.986 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:20.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:20.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.742201313000002, 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:20.992 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:20.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:20.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:20.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:20.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:20.998 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:20.998 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:20.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:20.999 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:20.923+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:21.000 INFO:teuthology.orchestra.run.smithi170.stdout:1046 read 0x4b05600 thru 0x4b05c1d (0x61e bytes) 2024-09-21T23:15:21.000 INFO:teuthology.orchestra.run.smithi170.stdout:1047 read 0x1969200 thru 0x19720a3 (0x8ea4 bytes) 2024-09-21T23:15:21.000 INFO:teuthology.orchestra.run.smithi170.stdout:1048 write 0x9622000 thru 0x962a0a4 (0x80a5 bytes) 2024-09-21T23:15:21.000 INFO:teuthology.orchestra.run.smithi170.stdout:1050 punch from 0x539b600 to 0x539f800, (0x4200 bytes) 2024-09-21T23:15:21.000 INFO:teuthology.orchestra.run.smithi170.stdout:1054 write 0xc855c00 thru 0xc85ab5b (0x4f5c bytes) 2024-09-21T23:15:21.000 INFO:teuthology.orchestra.run.smithi170.stdout:1055 read 0x5929e00 thru 0x592d245 (0x3446 bytes) 2024-09-21T23:15:21.000 INFO:teuthology.orchestra.run.smithi170.stdout:1058 write 0x834c600 thru 0x8355cac (0x96ad bytes) 2024-09-21T23:15:21.008 INFO:teuthology.orchestra.run.smithi170.stdout:1059 trunc from 0xc85ac00 to 0x6b1c400 2024-09-21T23:15:21.030 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:21.028+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:21.081 INFO:teuthology.orchestra.run.smithi170.stdout:1062 punch from 0x331a800 to 0x3329400, (0xec00 bytes) 2024-09-21T23:15:21.130 INFO:teuthology.orchestra.run.smithi170.stdout:1064 write 0xd2d8600 thru 0xd2d86aa (0xab bytes) 2024-09-21T23:15:21.131 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:21.123+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:21.133 INFO:teuthology.orchestra.run.smithi170.stdout:1066 read 0xc309a00 thru 0xc318eb2 (0xf4b3 bytes) 2024-09-21T23:15:21.133 INFO:teuthology.orchestra.run.smithi170.stdout:1068 write 0x56a6c00 thru 0x56b6987 (0xfd88 bytes) 2024-09-21T23:15:21.134 INFO:teuthology.orchestra.run.smithi170.stdout:1070 read 0x2ebba00 thru 0x2ec4a07 (0x9008 bytes) 2024-09-21T23:15:21.134 INFO:teuthology.orchestra.run.smithi170.stdout:1072 read 0xb722800 thru 0xb72f9e5 (0xd1e6 bytes) 2024-09-21T23:15:21.134 INFO:teuthology.orchestra.run.smithi170.stdout:1073 punch from 0x68b0600 to 0x68bfe00, (0xf800 bytes) 2024-09-21T23:15:21.136 INFO:teuthology.orchestra.run.smithi170.stdout:1074 punch from 0x3127a00 to 0x3128000, (0x600 bytes) 2024-09-21T23:15:21.136 INFO:teuthology.orchestra.run.smithi170.stdout:1082 read 0xac9ae00 thru 0xaca64e4 (0xb6e5 bytes) 2024-09-21T23:15:21.136 INFO:teuthology.orchestra.run.smithi170.stdout:1083 write 0x56bf000 thru 0x56c5809 (0x680a bytes) 2024-09-21T23:15:21.140 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:21.141 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:21.141 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:21.141 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:21.157 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:21.159 INFO:teuthology.orchestra.run.smithi170.stdout:1084 trunc from 0xd2d8800 to 0x180c800 2024-09-21T23:15:21.231 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:21.228+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:21.257 INFO:teuthology.orchestra.run.smithi170.stdout:1085 read 0x1ed200 thru 0x1f6f03 (0x9d04 bytes) 2024-09-21T23:15:21.302 INFO:teuthology.orchestra.run.smithi170.stdout:1087 write 0x98d0000 thru 0x98d5f60 (0x5f61 bytes) 2024-09-21T23:15:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:21.324 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:21.324 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:21.324 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:21.332 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:21.327+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:21.339 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:21.344 INFO:teuthology.orchestra.run.smithi170.stdout:1089 write 0xbf63200 thru 0xbf70bdb (0xd9dc bytes) 2024-09-21T23:15:21.347 INFO:teuthology.orchestra.run.smithi170.stdout:1092 read 0x300d400 thru 0x300e525 (0x1126 bytes) 2024-09-21T23:15:21.347 INFO:teuthology.orchestra.run.smithi170.stdout:1093 read 0x2be3a00 thru 0x2bec88f (0x8e90 bytes) 2024-09-21T23:15:21.347 INFO:teuthology.orchestra.run.smithi170.stdout:1094 write 0x92f9e00 thru 0x92fb806 (0x1a07 bytes) 2024-09-21T23:15:21.347 INFO:teuthology.orchestra.run.smithi170.stdout:1096 punch from 0x3100200 to 0x310b000, (0xae00 bytes) 2024-09-21T23:15:21.348 INFO:teuthology.orchestra.run.smithi170.stdout:1097 punch from 0x23a6e00 to 0x23a7c00, (0xe00 bytes) 2024-09-21T23:15:21.353 INFO:teuthology.orchestra.run.smithi170.stdout:1098 trunc from 0xbf70c00 to 0x9cb1a00 2024-09-21T23:15:21.433 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:21.432+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:21.454 INFO:teuthology.orchestra.run.smithi170.stdout:1103 trunc from 0x9cb1a00 to 0x24dc800 2024-09-21T23:15:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:1104 punch from 0x126d400 to 0x126f000, (0x1c00 bytes) 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:02.307304+0000", 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.211532582, 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073089369999999997, 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 7 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307304+0000", 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307304+0000", 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307307+0000", 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1550000000000001e-06 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307318+0000", 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1813e-05 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307340+0000", 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1772000000000001e-05 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307351+0000", 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1236000000000001e-05 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307413+0000", 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1692999999999994e-05 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.307418+0000", 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6759999999999998e-06 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.311777+0000", 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043591710000000002 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.311893+0000", 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000116267 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.312014+0000", 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012133099999999999 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.312713+0000", 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00069857299999999997 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.314506+0000", 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001792884 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.314551+0000", 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4879999999999997e-05 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:02.314613+0000", 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2485999999999998e-05 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.199060544000002, 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076817589999999998, 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 9 2024-09-21T23:15:21.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319781+0000", 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3529999999999997e-06 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319821+0000", 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0571999999999997e-05 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319871+0000", 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0114999999999997e-05 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319900+0000", 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8802999999999998e-05 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320073+0000", 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000172669 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320090+0000", 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7223000000000001e-05 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325068+0000", 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049774889999999999 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325233+0000", 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016537600000000001 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325491+0000", 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025775700000000001 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.326967+0000", 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001476684 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327236+0000", 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026841300000000001 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327389+0000", 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0001528 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327458+0000", 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9505000000000006e-05 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2008 3.3 3:c4d74490:::rbd_data.10f16a447f06.00000000000000b1:head [set-alloc-hint object_size 524288 write_size 524288,write 354304~14336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:21.536 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.134862933999999, 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088517769999999999, 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2008 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:21.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383977+0000", 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.959e-06 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384014+0000", 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6659999999999998e-05 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384039+0000", 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5745000000000002e-05 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384057+0000", 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7575999999999999e-05 2024-09-21T23:15:21.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384162+0000", 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010501 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384169+0000", 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390619+0000", 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0064506559999999999 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390698+0000", 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8777999999999997e-05 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390782+0000", 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3552000000000002e-05 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.391690+0000", 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000908609 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392719+0000", 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001029051 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392782+0000", 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2644999999999994e-05 2024-09-21T23:15:21.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392826+0000", 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3718999999999999e-05 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2086 3.9 3:96029272:::rbd_data.10f16a447f06.0000000000000064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:21.543 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.408404+0000", 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.110433085, 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0064494410000000002, 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2086 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408404+0000", 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408404+0000", 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408404+0000", 2024-09-21T23:15:21.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:15:21.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408408+0000", 2024-09-21T23:15:21.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2049999999999998e-06 2024-09-21T23:15:21.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408427+0000", 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.948e-05 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408433+0000", 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4049999999999997e-06 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.409699+0000", 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012656010000000001 2024-09-21T23:15:21.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.409760+0000", 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1036000000000003e-05 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.409823+0000", 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3170999999999995e-05 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.412103+0000", 2024-09-21T23:15:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0022801800000000001 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.414771+0000", 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026677110000000001 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.414813+0000", 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1638999999999999e-05 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.414853+0000", 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0515000000000001e-05 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2087 3.11 3:88303620:::rbd_data.10f16a447f06.0000000000000065:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.408534+0000", 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.110302710999999, 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0062284929999999999, 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.550 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2087 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408534+0000", 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408534+0000", 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408535+0000", 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8899999999999998e-07 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408540+0000", 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3000000000000001e-06 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408553+0000", 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2309e-05 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.408561+0000", 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8099999999999998e-06 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.409387+0000", 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00082684299999999996 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.409457+0000", 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9319999999999994e-05 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.409528+0000", 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1156999999999997e-05 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.412153+0000", 2024-09-21T23:15:21.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026246580000000002 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.414678+0000", 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002525574 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.414726+0000", 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7774e-05 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.414763+0000", 2024-09-21T23:15:21.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6859000000000001e-05 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2369 3.3 3:c601a63f:::rbd_data.10f16a447f06.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 215040~18944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.513834+0000", 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.005002573000001, 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060122420000000001, 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2369 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513834+0000", 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513834+0000", 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513836+0000", 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4139999999999999e-06 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513859+0000", 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2918e-05 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513881+0000", 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2439e-05 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513892+0000", 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1086e-05 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.514030+0000", 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013766700000000001 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.514322+0000", 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029210899999999998 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.514655+0000", 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033335599999999998 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.516245+0000", 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001589331 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.519638+0000", 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033930010000000001 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.519755+0000", 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011708 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.519847+0000", 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1841000000000004e-05 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.561 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4405.0:2371 3.15 e28/25 3:afa0cd39:::rbd_data.10f16a447f06.0000000000000019:head v 28'3, mlcod=27'1)", 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.003087304999999, 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076557819999999999, 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515751+0000", 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.547e-06 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515784+0000", 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2756000000000003e-05 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515810+0000", 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6259999999999999e-05 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515827+0000", 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6846000000000001e-05 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515987+0000", 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159729 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.516072+0000", 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4994999999999995e-05 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523367+0000", 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0072950250000000001 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523405+0000", 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8624000000000002e-05 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6064 3.9 3:9316769a:::rbd_data.10f1e443a6fa.000000000000002b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.482217+0000", 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.0366194999999996, 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0057886750000000001, 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.566 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6064 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482217+0000", 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482217+0000", 2024-09-21T23:15:21.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482220+0000", 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.39e-06 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482232+0000", 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2369e-05 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482270+0000", 2024-09-21T23:15:21.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7868999999999999e-05 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482300+0000", 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9802999999999999e-05 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482470+0000", 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017029600000000001 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482679+0000", 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000209271 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482931+0000", 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025158400000000001 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.484524+0000", 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0015932240000000001 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.487770+0000", 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003245501 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.487896+0000", 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012619399999999999 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.488006+0000", 2024-09-21T23:15:21.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011017400000000001 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6066 3.1b 3:db3973d4:::rbd_data.10f1e443a6fa.000000000000002c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.482883+0000", 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.0359535729999996, 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.006146777, 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6066 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482883+0000", 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482883+0000", 2024-09-21T23:15:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482886+0000", 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.153e-06 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482897+0000", 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1559999999999999e-05 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482932+0000", 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4925000000000002e-05 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.482954+0000", 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2480000000000002e-05 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.483168+0000", 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000213148 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.483367+0000", 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000199865 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.488332+0000", 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049646760000000003 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.488477+0000", 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014532 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489030+0000", 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00055265000000000002 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6113 3.15 3:a8204e3e:::rbd_data.10f1e443a6fa.000000000000003d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.489058+0000", 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.0297792799999996, 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070043420000000002, 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.577 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6113 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489058+0000", 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489058+0000", 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489059+0000", 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.796e-06 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489069+0000", 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4900000000000006e-06 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489117+0000", 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8523000000000003e-05 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489136+0000", 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8868999999999998e-05 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489270+0000", 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013335300000000001 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489488+0000", 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021858899999999999 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495680+0000", 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0061917839999999997 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495836+0000", 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000155947 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.496062+0000", 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022599100000000001 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6126 3.3 3:c61769d8:::rbd_data.10f1e443a6fa.000000000000004d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.489889+0000", 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.0289481800000004, 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0067089619999999997, 2024-09-21T23:15:21.582 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6126 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489889+0000", 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489889+0000", 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489891+0000", 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.373e-06 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489904+0000", 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.272e-05 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489938+0000", 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3852e-05 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.489960+0000", 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1878999999999998e-05 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490163+0000", 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000202969 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490348+0000", 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018541900000000001 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.496402+0000", 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060544520000000001 2024-09-21T23:15:21.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.496481+0000", 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8547999999999994e-05 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.496598+0000", 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011675000000000001 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6128 3.1 3:81c74549:::rbd_data.10f1e443a6fa.000000000000004e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.490316+0000", 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.0285210879999998, 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0056410119999999999, 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6128 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490316+0000", 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490316+0000", 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490318+0000", 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.807e-06 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490326+0000", 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8130000000000002e-06 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490347+0000", 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0716999999999998e-05 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490361+0000", 2024-09-21T23:15:21.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3625000000000001e-05 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490491+0000", 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013053699999999999 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.490693+0000", 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020124499999999999 2024-09-21T23:15:21.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495562+0000", 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0048696549999999996 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495728+0000", 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000166222 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495957+0000", 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022839099999999999 2024-09-21T23:15:21.592 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6155 3.1a 3:58809d19:::rbd_data.10f1e443a6fa.000000000000005f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.0251750829999997, 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0075748489999999998, 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6155 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:21.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493664+0000", 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0470000000000001e-06 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493674+0000", 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7659999999999993e-06 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493696+0000", 2024-09-21T23:15:21.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717e-05 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493717+0000", 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1002e-05 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495059+0000", 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0013413570000000001 2024-09-21T23:15:21.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495322+0000", 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026359099999999998 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495706+0000", 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038393999999999999 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.496952+0000", 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001245833 2024-09-21T23:15:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.500994+0000", 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0040416849999999997 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.501122+0000", 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128541 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.501237+0000", 2024-09-21T23:15:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011437 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6505 3.14 3:2adf2393:::rbd_data.10f1e443a6fa.0000000000000010:head [write 352256~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.664776+0000", 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.8540608919999997, 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059159989999999999, 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6505 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.664776+0000", 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.664776+0000", 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.664777+0000", 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6299999999999993e-07 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.664793+0000", 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6113e-05 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.664808+0000", 2024-09-21T23:15:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4450000000000001e-05 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.664817+0000", 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3389999999999999e-06 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.664936+0000", 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000118991 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.665174+0000", 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023783800000000001 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.665509+0000", 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000335168 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.668088+0000", 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0025791669999999998 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.670544+0000", 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024560799999999998 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.670629+0000", 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4513999999999998e-05 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.670692+0000", 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3375999999999998e-05 2024-09-21T23:15:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:8469 3.14 3:2c7e240f:::rbd_data.10f1e443a6fa.0000000000000006:head [write 253952~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:17.407309+0000", 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.1115278890000004, 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059944769999999998, 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 8469 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407309+0000", 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407309+0000", 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407310+0000", 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.223e-06 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407336+0000", 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6159000000000001e-05 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407357+0000", 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1053e-05 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407374+0000", 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6263e-05 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407490+0000", 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000116134 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.407684+0000", 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019400900000000001 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.408046+0000", 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036173800000000001 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.410120+0000", 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020742149999999999 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.413016+0000", 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002896343 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.413189+0000", 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017242200000000001 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.413304+0000", 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114918 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:8828 3.15 3:a9c60822:::rbd_data.10f1e443a6fa.0000000000000000:head [write 163840~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:17.540683+0000", 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.9781542380000001, 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0056952879999999997, 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 8828 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.540683+0000", 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.540683+0000", 2024-09-21T23:15:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.540685+0000", 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2199999999999999e-06 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.540715+0000", 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0312e-05 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.540745+0000", 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0032999999999999e-05 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.540765+0000", 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9772999999999999e-05 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.540918+0000", 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000152499 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.541315+0000", 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039715099999999999 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.546195+0000", 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0048803980000000002 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.546270+0000", 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4688e-05 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.546378+0000", 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108214 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:9909 3.1 3:82d50a28:::rbd_data.10f1e443a6fa.00000000000000a3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:17.891074+0000", 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.627762572, 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0056708100000000001, 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 9909 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891074+0000", 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891074+0000", 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891076+0000", 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.454e-06 2024-09-21T23:15:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891108+0000", 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2444999999999999e-05 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891126+0000", 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7572999999999999e-05 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891139+0000", 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2723e-05 2024-09-21T23:15:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891232+0000", 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3385000000000001e-05 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.891434+0000", 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020178500000000001 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.896598+0000", 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0051643710000000001 2024-09-21T23:15:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.896652+0000", 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4237000000000003e-05 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:17.896745+0000", 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2837000000000006e-05 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:10626 3.15 3:a9037f46:::rbd_data.10f1e443a6fa.000000000000003a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:18.163283+0000", 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.3555539379999999, 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0062410720000000003, 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10626 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163283+0000", 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163283+0000", 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163284+0000", 2024-09-21T23:15:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3799999999999999e-06 2024-09-21T23:15:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163293+0000", 2024-09-21T23:15:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4670000000000004e-06 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163317+0000", 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4068999999999999e-05 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163335+0000", 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8156999999999999e-05 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163755+0000", 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041954299999999998 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.164420+0000", 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00066561400000000003 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.169208+0000", 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0047878729999999998 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.169339+0000", 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013096000000000001 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.169524+0000", 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185009 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:10629 3.15 3:a8204e3e:::rbd_data.10f1e443a6fa.000000000000003d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:18.163457+0000", 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.355379938, 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0063839999999999999, 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10629 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163457+0000", 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163457+0000", 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163458+0000", 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.403e-06 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163465+0000", 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163487+0000", 2024-09-21T23:15:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1466000000000001e-05 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.163503+0000", 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6512999999999999e-05 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.166040+0000", 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0025366849999999999 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.166253+0000", 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000212728 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.169539+0000", 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0032862500000000001 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.169616+0000", 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7274000000000006e-05 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:18.169841+0000", 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022469100000000001 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:17960 3.1a 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head [set-alloc-hint object_size 524288 write_size 524288,write 131072~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:21.158674+0000", 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.36016341499999999, 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0056534630000000001, 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17960 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.158674+0000", 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.158674+0000", 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.158674+0000", 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7000000000000003e-07 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.158691+0000", 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6846000000000001e-05 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.158708+0000", 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6710000000000001e-05 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.158721+0000", 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3033e-05 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.159489+0000", 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000768398 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.159554+0000", 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4671000000000005e-05 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.159663+0000", 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010901 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.160911+0000", 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012476410000000001 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.164256+0000", 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033449690000000002 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.164296+0000", 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0317e-05 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:21.164327+0000", 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0997999999999997e-05 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:21.639 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:21.640 INFO:teuthology.orchestra.run.smithi170.stdout:1105 write 0xb547000 thru 0xb556983 (0xf984 bytes) 2024-09-21T23:15:21.640 INFO:teuthology.orchestra.run.smithi170.stdout:1107 read 0x7cca400 thru 0x7cd5685 (0xb286 bytes) 2024-09-21T23:15:21.640 INFO:teuthology.orchestra.run.smithi170.stdout:1108 write 0x51fb000 thru 0x52072e6 (0xc2e7 bytes) 2024-09-21T23:15:21.641 INFO:teuthology.orchestra.run.smithi170.stdout:1112 punch from 0x568ea00 to 0x569a200, (0xb800 bytes) 2024-09-21T23:15:21.641 INFO:teuthology.orchestra.run.smithi170.stdout:1113 read 0x1a45800 thru 0x1a4b83c (0x603d bytes) 2024-09-21T23:15:21.641 INFO:teuthology.orchestra.run.smithi170.stdout:1115 clone 3 order 21 su 1048576 sc 10 2024-09-21T23:15:21.641 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:21.642 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:21.636+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:21.741 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:21.740+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:21.772 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:21.772 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:21.772 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:21.772 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:21.786 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:21.841 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:21.835+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:21.913 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:21.913 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:21.927 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:21.971 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:21.967+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:22.043 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:22.040+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:22.096 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:275 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.142337259, 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013739622, 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 275 2024-09-21T23:15:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949268+0000", 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949270+0000", 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3120000000000001e-06 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949274+0000", 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9539999999999998e-06 2024-09-21T23:15:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949277+0000", 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2169999999999999e-06 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.962538+0000", 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013261545 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.962747+0000", 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020921199999999999 2024-09-21T23:15:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963008+0000", 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026011900000000001 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:276 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.142324478999999, 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014002473, 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 276 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949282+0000", 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.26e-06 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949286+0000", 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9230000000000004e-06 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949289+0000", 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963078+0000", 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013788804999999999 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963169+0000", 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1114999999999998e-05 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963283+0000", 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011409 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:277 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.142311995, 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014347459999999999, 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 277 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949294+0000", 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949295+0000", 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2580000000000001e-06 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949299+0000", 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949302+0000", 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9859999999999999e-06 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963339+0000", 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014037304 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963466+0000", 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012664600000000001 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963641+0000", 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017508399999999999 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:278 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.142299602, 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014599439000000001, 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 278 2024-09-21T23:15:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949307+0000", 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.139e-06 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949311+0000", 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0230000000000003e-06 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949314+0000", 2024-09-21T23:15:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963697+0000", 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014382544000000001 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963785+0000", 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7587e-05 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963905+0000", 2024-09-21T23:15:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012056 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:279 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.14228675, 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014999721000000001, 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 279 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949319+0000", 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0600000000000001e-07 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949320+0000", 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.313e-06 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949324+0000", 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9570000000000004e-06 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949327+0000", 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963958+0000", 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014630341 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964127+0000", 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016937300000000001 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964318+0000", 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019130699999999999 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:280 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.142252915, 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015213915, 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 280 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949353+0000", 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949361+0000", 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7709999999999995e-06 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949367+0000", 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.003e-06 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949371+0000", 2024-09-21T23:15:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964367+0000", 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014995746000000001 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964460+0000", 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3565000000000003e-05 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964566+0000", 2024-09-21T23:15:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106067 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:349 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 366592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.124897941, 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013723654, 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 349 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966707+0000", 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966708+0000", 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8200000000000001e-07 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966710+0000", 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0420000000000001e-06 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966714+0000", 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.177e-06 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966722+0000", 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2099999999999993e-06 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980309+0000", 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013586765000000001 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980361+0000", 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1643e-05 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980431+0000", 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0434999999999999e-05 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:350 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.124878637, 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013944434, 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 350 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.65e-07 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966728+0000", 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3069999999999999e-06 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966732+0000", 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9040000000000002e-06 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966735+0000", 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7819999999999998e-06 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980479+0000", 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013744528000000001 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980562+0000", 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2218999999999993e-05 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980671+0000", 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109429 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:351 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.124866534000001, 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014095582000000001, 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 351 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9799999999999999e-07 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966740+0000", 2024-09-21T23:15:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966744+0000", 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966747+0000", 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.117e-06 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980710+0000", 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013963074000000001 2024-09-21T23:15:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980762+0000", 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1779e-05 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980834+0000", 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2538000000000002e-05 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:352 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.124826964, 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014313115, 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 352 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966779+0000", 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.22e-07 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966783+0000", 2024-09-21T23:15:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8099999999999997e-06 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966788+0000", 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.758e-06 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966791+0000", 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1489999999999998e-06 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980882+0000", 2024-09-21T23:15:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014090965 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980969+0000", 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6222000000000002e-05 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981091+0000", 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000122889 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:353 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.124809455999999, 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014443615, 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 353 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966802+0000", 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9070000000000004e-06 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966806+0000", 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1200000000000004e-06 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966813+0000", 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5239999999999997e-06 2024-09-21T23:15:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981114+0000", 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014301385 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981166+0000", 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2021999999999998e-05 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981239+0000", 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3385000000000003e-05 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:972 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.944293865000001, 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013606012000000001, 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 972 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147311+0000", 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147312+0000", 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147313+0000", 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.243e-06 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147317+0000", 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.216e-06 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147321+0000", 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5149999999999998e-06 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160753+0000", 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013432062 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160825+0000", 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.216e-05 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160917+0000", 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2522000000000006e-05 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:973 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.944280381, 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013764903, 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 973 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147325+0000", 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8999999999999998e-07 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147327+0000", 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.46e-06 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147331+0000", 2024-09-21T23:15:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2339999999999996e-06 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147334+0000", 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6109999999999998e-06 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.160954+0000", 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013619229 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161011+0000", 2024-09-21T23:15:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7046000000000002e-05 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161090+0000", 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9033e-05 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:974 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 520192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.944266613, 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013957262999999999, 2024-09-21T23:15:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 974 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147340+0000", 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2589999999999999e-06 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147344+0000", 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1899999999999997e-06 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147348+0000", 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161133+0000", 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013784597000000001 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161205+0000", 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2692000000000001e-05 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161296+0000", 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0550000000000005e-05 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2367 3.16 3:6d83b35b:::rbd_object_map.10f16a447f06:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.578255503999999, 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015679904000000001, 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2367 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513351+0000", 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513355+0000", 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5929999999999999e-06 2024-09-21T23:15:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513372+0000", 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7062999999999999e-05 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513386+0000", 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3293e-05 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523249+0000", 2024-09-21T23:15:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.009863251 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523437+0000", 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018793 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.524029+0000", 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059259600000000003 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.525598+0000", 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001569017 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528769+0000", 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003170754 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528893+0000", 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012397100000000001 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.529030+0000", 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013660899999999999 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2582 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 366592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.537003448, 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013589129, 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2582 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554602+0000", 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1299999999999999e-07 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554603+0000", 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0550000000000001e-06 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554606+0000", 2024-09-21T23:15:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1429999999999998e-06 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554609+0000", 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6819999999999999e-06 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568014+0000", 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013405525 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568092+0000", 2024-09-21T23:15:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7180000000000003e-05 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568191+0000", 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9331000000000002e-05 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2583 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.536993218999999, 2024-09-21T23:15:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013835072, 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2583 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1e-07 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554613+0000", 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.212e-06 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554617+0000", 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554619+0000", 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6680000000000001e-06 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568239+0000", 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013620098000000001 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568331+0000", 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1255999999999995e-05 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568447+0000", 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011650399999999999 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2584 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.536982735, 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014048454, 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2584 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4999999999999999e-07 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554624+0000", 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.119e-06 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554627+0000", 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1200000000000002e-06 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554630+0000", 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7219999999999999e-06 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568490+0000", 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013860268 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568570+0000", 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9903999999999999e-05 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568671+0000", 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101071 2024-09-21T23:15:22.174 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2585 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.536972349999999, 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014283607, 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.175 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2585 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.77e-07 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554634+0000", 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.144e-06 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554638+0000", 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2440000000000002e-06 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554640+0000", 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6869999999999999e-06 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568717+0000", 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014077118 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568802+0000", 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4079000000000006e-05 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568917+0000", 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011495800000000001 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2586 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.536948325999999, 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014464238000000001, 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:22.179 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2586 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.28e-07 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554658+0000", 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.282e-06 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554662+0000", 2024-09-21T23:15:22.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5470000000000001e-06 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554665+0000", 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568947+0000", 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01428159 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569026+0000", 2024-09-21T23:15:22.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8950999999999997e-05 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569121+0000", 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5173000000000005e-05 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:22.183 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:22.184 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:22.179+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:22.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:22.285 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:22.284+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:22.319 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:22.319 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:22.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:22.387 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:22.379+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:22.487 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:22.483+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:22.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:22.573 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-21T23:15:22.588 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:22.583+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:22.640 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:22.640 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.293303658999999, 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:22.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:22.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:22.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:22.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:490 3.2 3:40fd9093:::rbd_data.10f16a447f06.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 389120~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.590533974, 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00426217, 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 490 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047969+0000", 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3329999999999999e-06 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048100+0000", 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013150299999999999 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048122+0000", 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2266e-05 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048136+0000", 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3125999999999999e-05 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048250+0000", 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114913 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048473+0000", 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022239199999999999 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048789+0000", 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031575700000000001 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.050278+0000", 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001489282 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052058+0000", 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001780479 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052148+0000", 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9300000000000002e-05 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052228+0000", 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0819e-05 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.652 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.525661297999999, 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:696 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'19, mlcod=27'14)", 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.525154705, 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042479270000000003, 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:22.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113347+0000", 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113355+0000", 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113373+0000", 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7226000000000001e-05 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113382+0000", 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7999999999999993e-06 2024-09-21T23:15:22.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114156+0000", 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077335100000000003 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114193+0000", 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6946999999999997e-05 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117516+0000", 2024-09-21T23:15:22.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033230299999999998 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117594+0000", 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7860000000000003e-05 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:2078 3.2 3:40fd9093:::rbd_data.10f16a447f06.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.226687834, 2024-09-21T23:15:22.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044032740000000004, 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2078 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:22.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411814+0000", 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.018e-06 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411820+0000", 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919999999999997e-06 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411838+0000", 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7943000000000001e-05 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411846+0000", 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9529999999999999e-06 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411973+0000", 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012721400000000001 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412087+0000", 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011305499999999999 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412223+0000", 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136427 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.413226+0000", 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001002635 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416125+0000", 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028995459999999998 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416176+0000", 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1347000000000002e-05 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416216+0000", 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9243999999999999e-05 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.1463512160000002, 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:22.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.0191342040000002, 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:22.673 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:22.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:22.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1043704730000004, 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:15:22.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6494177069999996, 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:22.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.7993913240000001, 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:15:22.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:15:22.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.799349906, 2024-09-21T23:15:22.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:15:22.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:15:22.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:15:22.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:15:22.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17577 3.7 e36/21 3:e1bff944:::rbd_object_map.10f1e443a6fa:head v 36'1278, mlcod=36'1276)", 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.942005+0000", 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.6964950509999999, 2024-09-21T23:15:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042841709999999998, 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942005+0000", 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942005+0000", 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942007+0000", 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6169999999999999e-06 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942018+0000", 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.116e-05 2024-09-21T23:15:22.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942046+0000", 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7613000000000001e-05 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942059+0000", 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3273e-05 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942164+0000", 2024-09-21T23:15:22.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105406 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942198+0000", 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4143000000000002e-05 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.946173+0000", 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039743590000000002 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.946289+0000", 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001166 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.6172639800000002, 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:15:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:15:22.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:15:22.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17787 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.055529+0000", 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.5829708839999999, 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043080560000000002, 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17787 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055529+0000", 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055529+0000", 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055531+0000", 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7680000000000001e-06 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055540+0000", 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3030000000000005e-06 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055563+0000", 2024-09-21T23:15:22.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2291000000000001e-05 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055580+0000", 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7248000000000002e-05 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055703+0000", 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000123261 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055913+0000", 2024-09-21T23:15:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209762 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.056181+0000", 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026784399999999999 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.058489+0000", 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002308381 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.059617+0000", 2024-09-21T23:15:22.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011275840000000001 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.059748+0000", 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013159000000000001 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.059837+0000", 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9023999999999993e-05 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17918 3.7 e36/21 3:e1bff944:::rbd_object_map.10f1e443a6fa:head v 36'1309, mlcod=36'1306)", 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.140256+0000", 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.4982439350000001, 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043485729999999997, 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140256+0000", 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140256+0000", 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140258+0000", 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140267+0000", 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4450000000000002e-06 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140287+0000", 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9797999999999999e-05 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140298+0000", 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0702e-05 2024-09-21T23:15:22.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140450+0000", 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015234299999999999 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140511+0000", 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0825000000000001e-05 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.144450+0000", 2024-09-21T23:15:22.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039394809999999999 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.144605+0000", 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015468900000000001 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.475321637, 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:15:22.723 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:15:22.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17960 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'104, mlcod=35'101)", 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163772+0000", 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.474728681, 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041931019999999998, 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163772+0000", 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163772+0000", 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163773+0000", 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5230000000000001e-06 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163814+0000", 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0531000000000002e-05 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163842+0000", 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7877000000000001e-05 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163859+0000", 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7512000000000001e-05 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.164374+0000", 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051483099999999999 2024-09-21T23:15:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.164423+0000", 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8911e-05 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167929+0000", 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035060099999999999 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167965+0000", 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5907e-05 2024-09-21T23:15:22.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17932 3.7 e36/21 3:e1bff944:::rbd_object_map.10f1e443a6fa:head v 36'1321, mlcod=36'1318)", 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.172262+0000", 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.466238414, 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041823110000000002, 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172262+0000", 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172262+0000", 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172263+0000", 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4799999999999997e-07 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172271+0000", 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8000000000000004e-06 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172304+0000", 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2713999999999999e-05 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172315+0000", 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0604e-05 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172415+0000", 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100389 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172838+0000", 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042283499999999999 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176318+0000", 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034796620000000001 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176444+0000", 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012655899999999999 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18444 3.4 3:2145313d:::rbd_data.10f1e443a6fa.000000000000017b:head [set-alloc-hint object_size 524288 write_size 524288,write 278528~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.268995088, 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046378829999999998, 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18444 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369508+0000", 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8569999999999999e-06 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369539+0000", 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1108000000000001e-05 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369580+0000", 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0679999999999997e-05 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369602+0000", 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1746000000000002e-05 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369735+0000", 2024-09-21T23:15:22.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133349 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369934+0000", 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000199507 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.370101+0000", 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166119 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.371736+0000", 2024-09-21T23:15:22.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016352059999999999 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.373849+0000", 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002113326 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.374015+0000", 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016577200000000001 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.374143+0000", 2024-09-21T23:15:22.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012821299999999999 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18770 3.0 3:00c22c69:::rbd_data.10f1e443a6fa.0000000000000168:head [set-alloc-hint object_size 524288 write_size 524288,write 155648~2560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.554452+0000", 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.084047889, 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043579869999999998, 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:22.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18770 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554452+0000", 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554452+0000", 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554455+0000", 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.103e-06 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554467+0000", 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.223e-05 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554498+0000", 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0944000000000001e-05 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554519+0000", 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1806000000000001e-05 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554665+0000", 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014598399999999999 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554959+0000", 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029349399999999999 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.555262+0000", 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030329800000000002 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.556551+0000", 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012884369999999999 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.558574+0000", 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002023427 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.558674+0000", 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9479000000000001e-05 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.558810+0000", 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136785 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.747 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:22.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:22.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:22.750 INFO:teuthology.orchestra.run.smithi170.stdout:1116 read 0x3af2a00 thru 0x3afa26c (0x786d bytes) 2024-09-21T23:15:22.750 INFO:teuthology.orchestra.run.smithi170.stdout:1119 read 0x2e02c00 thru 0x2e088f2 (0x5cf3 bytes) 2024-09-21T23:15:22.750 INFO:teuthology.orchestra.run.smithi170.stdout:1121 punch from 0x9126c00 to 0x9128200, (0x1600 bytes) 2024-09-21T23:15:22.750 INFO:teuthology.orchestra.run.smithi170.stdout:1124 trunc from 0xb556a00 to 0xdf4c200 2024-09-21T23:15:22.750 INFO:teuthology.orchestra.run.smithi170.stdout:1125 write 0x8bdce00 thru 0x8be8284 (0xb485 bytes) 2024-09-21T23:15:22.750 INFO:teuthology.orchestra.run.smithi170.stdout:1129 write 0xdbe5800 thru 0xdbecfa6 (0x77a7 bytes) 2024-09-21T23:15:22.751 INFO:teuthology.orchestra.run.smithi170.stdout:1131 write 0xcd3c800 thru 0xcd3d640 (0xe41 bytes) 2024-09-21T23:15:22.751 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:22.743+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:22.807 INFO:teuthology.orchestra.run.smithi170.stdout:1132 trunc from 0xdf4c200 to 0x1d5dc00 2024-09-21T23:15:22.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:15:22.880 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:22.880+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:22.893 INFO:teuthology.orchestra.run.smithi170.stdout:1134 trunc from 0x1d5dc00 to 0x9f5ee00 2024-09-21T23:15:22.916 INFO:teuthology.orchestra.run.smithi170.stdout:1138 write 0x9f09c00 thru 0x9f19a02 (0xfe03 bytes) 2024-09-21T23:15:22.917 INFO:teuthology.orchestra.run.smithi170.stdout:1139 read 0x14a7600 thru 0x14b29b7 (0xb3b8 bytes) 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4477.0:495 3.5 e37/21 3:a6a92186:::rbd_object_map.10f156ded9d7:head v 37'101, mlcod=37'99)", 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:22.927594+0000", 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0010719779999999999, 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011133510000000001, 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:22.927 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927594+0000", 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927594+0000", 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:22.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927595+0000", 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1299999999999999e-07 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927603+0000", 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4379999999999995e-06 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927617+0000", 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3635000000000001e-05 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927625+0000", 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7060000000000004e-06 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927673+0000", 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8353999999999998e-05 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:22.927689+0000", 2024-09-21T23:15:22.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6061000000000001e-05 2024-09-21T23:15:22.931 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.931 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:22.931 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.931 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:22.931 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:22.931 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-21T23:15:22.931 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:22.938 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:22.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:22.950 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:22.943+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:22.951 INFO:teuthology.orchestra.run.smithi170.stdout:1140 write 0xbd71e00 thru 0xbd78829 (0x6a2a bytes) 2024-09-21T23:15:22.955 INFO:teuthology.orchestra.run.smithi170.stdout:1141 read 0x449a400 thru 0x44a6cac (0xc8ad bytes) 2024-09-21T23:15:22.956 INFO:teuthology.orchestra.run.smithi170.stdout:1142 write 0xbfe600 thru 0xbff678 (0x1079 bytes) 2024-09-21T23:15:22.956 INFO:teuthology.orchestra.run.smithi170.stdout:1147 read 0x32df400 thru 0x32e86e8 (0x92e9 bytes) 2024-09-21T23:15:22.956 INFO:teuthology.orchestra.run.smithi170.stdout:1148 write 0x4b50c00 thru 0x4b564fe (0x58ff bytes) 2024-09-21T23:15:22.956 INFO:teuthology.orchestra.run.smithi170.stdout:1149 read 0x9f31800 thru 0x9f319c6 (0x1c7 bytes) 2024-09-21T23:15:22.957 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:22.957 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:22.979 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:22.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:22.992 INFO:teuthology.orchestra.run.smithi170.stdout:1150 write 0xc66e800 thru 0xc67b805 (0xd006 bytes) 2024-09-21T23:15:22.994 INFO:teuthology.orchestra.run.smithi170.stdout:1152 write 0x9111e00 thru 0x911f298 (0xd499 bytes) 2024-09-21T23:15:22.994 INFO:teuthology.orchestra.run.smithi170.stdout:1153 write 0xc5ed000 thru 0xc5fc646 (0xf647 bytes) 2024-09-21T23:15:22.998 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:23.021 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:23.021 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:23.022 INFO:teuthology.orchestra.run.smithi170.stdout:1154 trunc from 0xc67ba00 to 0x1180000 2024-09-21T23:15:23.046 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:23.047 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:23.051 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:23.051 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:23.051 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:23.043+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:23.081 INFO:teuthology.orchestra.run.smithi170.stdout:1156 read 0xb11a00 thru 0xb1d6a4 (0xbca5 bytes) 2024-09-21T23:15:23.083 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:23.083 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:23.097 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:23.142 INFO:teuthology.orchestra.run.smithi170.stdout:1157 write 0xd938600 thru 0xd9446a4 (0xc0a5 bytes) 2024-09-21T23:15:23.152 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:23.147+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:23.153 INFO:teuthology.orchestra.run.smithi170.stdout:1160 trunc from 0xd944800 to 0x9ede600 2024-09-21T23:15:23.195 INFO:teuthology.orchestra.run.smithi170.stdout:1161 read 0x12f0600 thru 0x12fac2d (0xa62e bytes) 2024-09-21T23:15:23.199 INFO:teuthology.orchestra.run.smithi170.stdout:1162 read 0x98ca200 thru 0x98d9d3b (0xfb3c bytes) 2024-09-21T23:15:23.199 INFO:teuthology.orchestra.run.smithi170.stdout:1163 read 0x1fbb200 thru 0x1fc276a (0x756b bytes) 2024-09-21T23:15:23.199 INFO:teuthology.orchestra.run.smithi170.stdout:1166 punch from 0x6c0f200 to 0x6c11400, (0x2200 bytes) 2024-09-21T23:15:23.200 INFO:teuthology.orchestra.run.smithi170.stdout:1168 read 0x2fd9000 thru 0x2fe528f (0xc290 bytes) 2024-09-21T23:15:23.200 INFO:teuthology.orchestra.run.smithi170.stdout:1169 read 0x8599800 thru 0x859f479 (0x5c7a bytes) 2024-09-21T23:15:23.200 INFO:teuthology.orchestra.run.smithi170.stdout:1171 read 0x73f5c00 thru 0x73f5cac (0xad bytes) 2024-09-21T23:15:23.203 INFO:teuthology.orchestra.run.smithi170.stdout:1173 trunc from 0x9ede600 to 0xaf00800 2024-09-21T23:15:23.234 INFO:teuthology.orchestra.run.smithi170.stdout:1174 trunc from 0xaf00800 to 0xd4da800 2024-09-21T23:15:23.252 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:23.252+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:23.263 INFO:teuthology.orchestra.run.smithi170.stdout:1175 punch from 0x6d800 to 0x7b200, (0xda00 bytes) 2024-09-21T23:15:23.266 INFO:teuthology.orchestra.run.smithi170.stdout:1177 write 0x9d2d600 thru 0x9d3d288 (0xfc89 bytes) 2024-09-21T23:15:23.275 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:23.275 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:23.291 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:23.306 INFO:teuthology.orchestra.run.smithi170.stdout:1178 write 0xe92d400 thru 0xe92f4a4 (0x20a5 bytes) 2024-09-21T23:15:23.308 INFO:teuthology.orchestra.run.smithi170.stdout:1179 write 0xdfa3000 thru 0xdfa7304 (0x4305 bytes) 2024-09-21T23:15:23.308 INFO:teuthology.orchestra.run.smithi170.stdout:1180 read 0xaa7a600 thru 0xaa7e5cb (0x3fcc bytes) 2024-09-21T23:15:23.308 INFO:teuthology.orchestra.run.smithi170.stdout:1183 write 0x15bc000 thru 0x15bf389 (0x338a bytes) 2024-09-21T23:15:23.339 INFO:teuthology.orchestra.run.smithi170.stdout:1184 trunc from 0xe92f600 to 0x6ef800 2024-09-21T23:15:23.353 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:23.352+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:23.388 INFO:teuthology.orchestra.run.smithi170.stdout:1185 read 0x44ea00 thru 0x45adec (0xc3ed bytes) 2024-09-21T23:15:23.404 INFO:teuthology.orchestra.run.smithi170.stdout:1188 trunc from 0x6ef800 to 0x68f1e00 2024-09-21T23:15:23.435 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:23.436 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:23.450 INFO:teuthology.orchestra.run.smithi170.stdout:1189 write 0x928cc00 thru 0x929450c (0x790d bytes) 2024-09-21T23:15:23.452 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:23.453 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:23.447+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:23.466 INFO:teuthology.orchestra.run.smithi170.stdout:1190 trunc from 0x9294600 to 0x3bdf600 2024-09-21T23:15:23.496 INFO:teuthology.orchestra.run.smithi170.stdout:1191 read 0x18cee00 thru 0x18db5e4 (0xc7e5 bytes) 2024-09-21T23:15:23.497 INFO:teuthology.orchestra.run.smithi170.stdout:1193 write 0x2bca200 thru 0x2bd9626 (0xf427 bytes) 2024-09-21T23:15:23.498 INFO:teuthology.orchestra.run.smithi170.stdout:1194 read 0x2a4e600 thru 0x2a4ea6e (0x46f bytes) 2024-09-21T23:15:23.498 INFO:teuthology.orchestra.run.smithi170.stdout:1196 read 0x14f5e00 thru 0x14f878a (0x298b bytes) 2024-09-21T23:15:23.498 INFO:teuthology.orchestra.run.smithi170.stdout:1197 trunc from 0x3bdf600 to 0x3eb0a00 2024-09-21T23:15:23.537 INFO:teuthology.orchestra.run.smithi170.stdout:1199 trunc from 0x3eb0a00 to 0xcd8bc00 2024-09-21T23:15:23.553 INFO:teuthology.orchestra.run.smithi170.stdout:1203 write 0x16e3a00 thru 0x16e48c2 (0xec3 bytes) 2024-09-21T23:15:23.554 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:23.552+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:23.577 INFO:teuthology.orchestra.run.smithi170.stdout:1206 write 0xd66a800 thru 0xd671a28 (0x7229 bytes) 2024-09-21T23:15:23.580 INFO:teuthology.orchestra.run.smithi170.stdout:1208 read 0x4ae0600 thru 0x4ae690a (0x630b bytes) 2024-09-21T23:15:23.580 INFO:teuthology.orchestra.run.smithi170.stdout:1209 punch from 0x9e56800 to 0x9e5c800, (0x6000 bytes) 2024-09-21T23:15:23.582 INFO:teuthology.orchestra.run.smithi170.stdout:1211 trunc from 0xd671c00 to 0xc807400 2024-09-21T23:15:23.594 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:23.594 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:23.610 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:23.630 INFO:teuthology.orchestra.run.smithi170.stdout:1213 write 0xcf8ac00 thru 0xcf95fe6 (0xb3e7 bytes) 2024-09-21T23:15:23.631 INFO:teuthology.orchestra.run.smithi170.stdout:1214 write 0x4505a00 thru 0x4506e37 (0x1438 bytes) 2024-09-21T23:15:23.632 INFO:teuthology.orchestra.run.smithi170.stdout:1215 read 0xa27e600 thru 0xa284ca5 (0x66a6 bytes) 2024-09-21T23:15:23.655 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:23.651+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:23.656 INFO:teuthology.orchestra.run.smithi170.stdout:1216 write 0xe202a00 thru 0xe20e860 (0xbe61 bytes) 2024-09-21T23:15:23.682 INFO:teuthology.orchestra.run.smithi170.stdout:1220 write 0xed09000 thru 0xed125ef (0x95f0 bytes) 2024-09-21T23:15:23.685 INFO:teuthology.orchestra.run.smithi170.stdout:1221 read 0x2db4800 thru 0x2dc21fe (0xd9ff bytes) 2024-09-21T23:15:23.685 INFO:teuthology.orchestra.run.smithi170.stdout:1223 punch from 0x7a9f200 to 0x7aa4200, (0x5000 bytes) 2024-09-21T23:15:23.686 INFO:teuthology.orchestra.run.smithi170.stdout:1224 punch from 0x41aca00 to 0x41b9400, (0xca00 bytes) 2024-09-21T23:15:23.686 INFO:teuthology.orchestra.run.smithi170.stdout:1228 write 0x7321c00 thru 0x732ba13 (0x9e14 bytes) 2024-09-21T23:15:23.687 INFO:teuthology.orchestra.run.smithi170.stdout:1234 write 0x7b8c00 thru 0x7c8ad3 (0xfed4 bytes) 2024-09-21T23:15:23.687 INFO:teuthology.orchestra.run.smithi170.stdout:1238 read 0xca36c00 thru 0xca3cff5 (0x63f6 bytes) 2024-09-21T23:15:23.687 INFO:teuthology.orchestra.run.smithi170.stdout:1239 write 0x8abac00 thru 0x8ac2ec1 (0x82c2 bytes) 2024-09-21T23:15:23.687 INFO:teuthology.orchestra.run.smithi170.stdout:1240 write 0x41b1c00 thru 0x41b89ed (0x6dee bytes) 2024-09-21T23:15:23.693 INFO:teuthology.orchestra.run.smithi170.stdout:1243 punch from 0x8ba5800 to 0x8ba7200, (0x1a00 bytes) 2024-09-21T23:15:23.694 INFO:teuthology.orchestra.run.smithi170.stdout:1244 clone 4 order 18 su 65536 sc 9 2024-09-21T23:15:23.747 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:23.747 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:23.755 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:23.752+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:23.761 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:23.810 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:15:23.876 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:23.876+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:23.936 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:23.937 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:23.939 INFO:teuthology.orchestra.run.smithi170.stdout:leaving image image_client.0-clone3 intact 2024-09-21T23:15:23.951 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:23.957 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:23.956+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:24.058 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:24.051+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:24.072 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:24.072 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:24.086 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:24.171 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:24.167+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:24.259 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:24.255+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:24.259 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:24.260 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:24.274 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:24.359 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:24.355+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:24.438 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:24.438 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:24.452 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:24.460 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:24.456+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:24.560 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:24.555+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:24.601 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-21T23:15:24.621 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:24.621 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:24.637 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:24.679 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:24.675+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:24.762 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:24.760+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:24.827 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:24.827 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:24.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:24.862 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:24.855+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:24.963 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:24.959+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:24.990 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:24.990 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:25.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:25.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:25.096 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:25.096+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:25.133 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:25.133 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:25.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:25.192 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:25.192+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:25.195 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:25.216 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:25.216 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:25.237 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:25.237 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:25.266 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:25.259+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:25.266 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:25.266 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:25.292 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:25.292 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:25.316 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:25.316 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:25.320 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:25.320 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:25.340 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.995269018999998, 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:25.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:490 3.2 3:40fd9093:::rbd_data.10f16a447f06.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 389120~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.292499333999999, 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00426217, 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 490 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047966+0000", 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.047969+0000", 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3329999999999999e-06 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048100+0000", 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013150299999999999 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048122+0000", 2024-09-21T23:15:25.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2266e-05 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048136+0000", 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3125999999999999e-05 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048250+0000", 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114913 2024-09-21T23:15:25.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048473+0000", 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022239199999999999 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.048789+0000", 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031575700000000001 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.050278+0000", 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001489282 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052058+0000", 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001780479 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052148+0000", 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9300000000000002e-05 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.052228+0000", 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0819e-05 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.352 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.227626657999998, 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:25.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:696 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'19, mlcod=27'14)", 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.227120065000001, 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042479270000000003, 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113346+0000", 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113347+0000", 2024-09-21T23:15:25.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113355+0000", 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113373+0000", 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7226000000000001e-05 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113382+0000", 2024-09-21T23:15:25.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7999999999999993e-06 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114156+0000", 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077335100000000003 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.114193+0000", 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6946999999999997e-05 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117516+0000", 2024-09-21T23:15:25.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033230299999999998 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117594+0000", 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7860000000000003e-05 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4405.0:2078 3.2 3:40fd9093:::rbd_data.10f16a447f06.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.928653193999999, 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044032740000000004, 2024-09-21T23:15:25.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4405", 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2078 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411812+0000", 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411814+0000", 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.018e-06 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411820+0000", 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919999999999997e-06 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411838+0000", 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7943000000000001e-05 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411846+0000", 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9529999999999999e-06 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.411973+0000", 2024-09-21T23:15:25.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012721400000000001 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412087+0000", 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011305499999999999 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.412223+0000", 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136427 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.413226+0000", 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001002635 2024-09-21T23:15:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416125+0000", 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028995459999999998 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416176+0000", 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1347000000000002e-05 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.416216+0000", 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9243999999999999e-05 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.8483165760000002, 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:25.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:25.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:25.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7210995639999993, 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:25.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:25.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.8063358330000003, 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3513830670000004, 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.384 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:25.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.389 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5013566840000001, 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:15:25.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:25.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5013152659999998, 2024-09-21T23:15:25.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:15:25.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:15:25.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17577 3.7 e36/21 3:e1bff944:::rbd_object_map.10f1e443a6fa:head v 36'1278, mlcod=36'1276)", 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.942005+0000", 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3984604110000003, 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042841709999999998, 2024-09-21T23:15:25.401 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942005+0000", 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942005+0000", 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942007+0000", 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6169999999999999e-06 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942018+0000", 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.116e-05 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942046+0000", 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7613000000000001e-05 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942059+0000", 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3273e-05 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942164+0000", 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105406 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.942198+0000", 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4143000000000002e-05 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.946173+0000", 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039743590000000002 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.946289+0000", 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001166 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3192293399999997, 2024-09-21T23:15:25.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17787 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.055529+0000", 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.2849362439999998, 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043080560000000002, 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17787 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055529+0000", 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055529+0000", 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055531+0000", 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7680000000000001e-06 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055540+0000", 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3030000000000005e-06 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055563+0000", 2024-09-21T23:15:25.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2291000000000001e-05 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055580+0000", 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7248000000000002e-05 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055703+0000", 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000123261 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.055913+0000", 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209762 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.056181+0000", 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026784399999999999 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.058489+0000", 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002308381 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.059617+0000", 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011275840000000001 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.059748+0000", 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013159000000000001 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.059837+0000", 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9023999999999993e-05 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17918 3.7 e36/21 3:e1bff944:::rbd_object_map.10f1e443a6fa:head v 36'1309, mlcod=36'1306)", 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.140256+0000", 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.2002092949999996, 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043485729999999997, 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140256+0000", 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140256+0000", 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140258+0000", 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2899999999999999e-06 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140267+0000", 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4450000000000002e-06 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140287+0000", 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9797999999999999e-05 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140298+0000", 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0702e-05 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140450+0000", 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015234299999999999 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.140511+0000", 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0825000000000001e-05 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.144450+0000", 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039394809999999999 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.144605+0000", 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015468900000000001 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.1772869970000004, 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.421 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:15:25.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17960 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'104, mlcod=35'101)", 2024-09-21T23:15:25.425 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163772+0000", 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.1766940410000002, 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041931019999999998, 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163772+0000", 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163772+0000", 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163773+0000", 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5230000000000001e-06 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163814+0000", 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0531000000000002e-05 2024-09-21T23:15:25.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163842+0000", 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7877000000000001e-05 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163859+0000", 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7512000000000001e-05 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.164374+0000", 2024-09-21T23:15:25.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051483099999999999 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.164423+0000", 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8911e-05 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167929+0000", 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035060099999999999 2024-09-21T23:15:25.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167965+0000", 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5907e-05 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17932 3.7 e36/21 3:e1bff944:::rbd_object_map.10f1e443a6fa:head v 36'1321, mlcod=36'1318)", 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.172262+0000", 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.1682037740000002, 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041823110000000002, 2024-09-21T23:15:25.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172262+0000", 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172262+0000", 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172263+0000", 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4799999999999997e-07 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172271+0000", 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8000000000000004e-06 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172304+0000", 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2713999999999999e-05 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172315+0000", 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0604e-05 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172415+0000", 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100389 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172838+0000", 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042283499999999999 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176318+0000", 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034796620000000001 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176444+0000", 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012655899999999999 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18444 3.4 3:2145313d:::rbd_data.10f1e443a6fa.000000000000017b:head [set-alloc-hint object_size 524288 write_size 524288,write 278528~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.970960448, 2024-09-21T23:15:25.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046378829999999998, 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18444 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369508+0000", 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8569999999999999e-06 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369539+0000", 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1108000000000001e-05 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369580+0000", 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0679999999999997e-05 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369602+0000", 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1746000000000002e-05 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369735+0000", 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133349 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369934+0000", 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000199507 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.370101+0000", 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166119 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.371736+0000", 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016352059999999999 2024-09-21T23:15:25.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.373849+0000", 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002113326 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.374015+0000", 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016577200000000001 2024-09-21T23:15:25.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.374143+0000", 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012821299999999999 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18770 3.0 3:00c22c69:::rbd_data.10f1e443a6fa.0000000000000168:head [set-alloc-hint object_size 524288 write_size 524288,write 155648~2560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.554452+0000", 2024-09-21T23:15:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.7860132489999998, 2024-09-21T23:15:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043579869999999998, 2024-09-21T23:15:25.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18770 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554452+0000", 2024-09-21T23:15:25.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554452+0000", 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554455+0000", 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.103e-06 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554467+0000", 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.223e-05 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554498+0000", 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0944000000000001e-05 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554519+0000", 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1806000000000001e-05 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554665+0000", 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014598399999999999 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.554959+0000", 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029349399999999999 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.555262+0000", 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030329800000000002 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.556551+0000", 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012884369999999999 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.558574+0000", 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002023427 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.558674+0000", 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9479000000000001e-05 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.558810+0000", 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136785 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.447 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:25.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:25.449 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:25.443+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:25.549 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:25.548+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:25.588 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-21T23:15:25.589 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:15:25.589 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-21T23:15:25.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-21T23:15:25.650 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:25.650 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:25.650 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:25.650 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:25.650 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:25.643+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:25.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:25.740 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:25.744 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:25.744 INFO:tasks.thrashosds.thrasher:Failed to reset_purged_snaps_last, ignoring 2024-09-21T23:15:25.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-21T23:15:25.750 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:25.743+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:25.814 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:25.814 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:25.814 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:25.814 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:25.814 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:25.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:25.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:15:25.880 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:25.880+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:25.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-21T23:15:25.969 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:25.968+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17932 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8311011270000002, 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041236575999999997, 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17932 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138423+0000", 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138428+0000", 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7929999999999997e-06 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138438+0000", 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0672999999999999e-05 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138446+0000", 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4200000000000001e-06 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.171369+0000", 2024-09-21T23:15:25.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032922699999999999 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.171556+0000", 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000187071 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172087+0000", 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053124200000000002 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179510+0000", 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074233470000000003 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179565+0000", 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5047999999999999e-05 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179617+0000", 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1579999999999997e-05 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179659+0000", 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1949000000000001e-05 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17933 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8310677450000004, 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041549861, 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17933 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:25.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138461+0000", 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.33e-06 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138470+0000", 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6239999999999997e-06 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138478+0000", 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.173660+0000", 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035182577 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.173815+0000", 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154759 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.174562+0000", 2024-09-21T23:15:25.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074730300000000005 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179805+0000", 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005243195 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179835+0000", 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9431e-05 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179918+0000", 2024-09-21T23:15:25.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3591999999999995e-05 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180005+0000", 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7033999999999994e-05 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17934 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8310303149999996, 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045630958999999999, 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17934 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138494+0000", 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.23e-07 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138498+0000", 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.374e-06 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138508+0000", 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0135999999999999e-05 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138518+0000", 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5140000000000007e-06 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176013+0000", 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.037495095999999999 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176109+0000", 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5759000000000003e-05 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176830+0000", 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072154200000000004 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180029+0000", 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031987000000000001 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.183898+0000", 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038689639999999999 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184014+0000", 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000116304 2024-09-21T23:15:25.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184124+0000", 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010984699999999999 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17935 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8309952379999999, 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045851705999999999, 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17935 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138529+0000", 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000002e-07 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138533+0000", 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8829999999999999e-06 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138558+0000", 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5123999999999999e-05 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138566+0000", 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1259999999999998e-06 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178007+0000", 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.039441213000000003 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178100+0000", 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2608000000000005e-05 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178652+0000", 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055207499999999996 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184230+0000", 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055785110000000004 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184245+0000", 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4958e-05 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184301+0000", 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5742000000000003e-05 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:25.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184380+0000", 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8832999999999996e-05 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17936 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.830944465, 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046903247000000002, 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:25.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17936 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:25.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138580+0000", 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138584+0000", 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5940000000000004e-06 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138594+0000", 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0316000000000001e-05 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138602+0000", 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.17e-06 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180186+0000", 2024-09-21T23:15:25.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041584401999999999 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180422+0000", 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023555099999999999 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.181094+0000", 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067216599999999997 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184518+0000", 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034243770000000001 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185304+0000", 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078624499999999998 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:25.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185406+0000", 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101691 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185482+0000", 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6081999999999994e-05 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.000 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17937 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8309069789999999, 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047158114000000001, 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17937 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138617+0000", 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2399999999999997e-07 2024-09-21T23:15:26.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138622+0000", 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5530000000000003e-06 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138632+0000", 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0096e-05 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138639+0000", 2024-09-21T23:15:26.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1640000000000004e-06 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.182451+0000", 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.043812140999999999 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.182571+0000", 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012012 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.183035+0000", 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046411899999999998 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185503+0000", 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024672840000000001 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185610+0000", 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107126 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185720+0000", 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010995 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185775+0000", 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4836999999999997e-05 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18434 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6182112120000003, 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040579337, 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18434 2024-09-21T23:15:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351313+0000", 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351317+0000", 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9260000000000002e-06 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351326+0000", 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0089999999999996e-06 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351333+0000", 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2169999999999997e-06 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.382673+0000", 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.031340149999999997 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.382938+0000", 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000264567 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.383955+0000", 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001017783 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391521+0000", 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075656059999999999 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391668+0000", 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014702399999999999 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391830+0000", 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000162447 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391891+0000", 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1033999999999999e-05 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18435 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.012 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6181809459999998, 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042792912000000002, 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18435 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:26.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351343+0000", 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.68e-07 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351347+0000", 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4569999999999998e-06 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351357+0000", 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0629999999999995e-06 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351365+0000", 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4800000000000001e-06 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.386383+0000", 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035018214999999998 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.386544+0000", 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016093999999999999 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.387525+0000", 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098102499999999995 2024-09-21T23:15:26.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392029+0000", 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045039879999999996 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.393961+0000", 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001931552 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394042+0000", 2024-09-21T23:15:26.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1255999999999996e-05 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394135+0000", 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3368000000000005e-05 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18436 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6181491059999997, 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044976601999999997, 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18436 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:26.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351375+0000", 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7100000000000002e-07 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351379+0000", 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351389+0000", 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351396+0000", 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.389268+0000", 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.037871888999999999 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.389448+0000", 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018001900000000001 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.390406+0000", 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095752800000000002 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396081+0000", 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0056750050000000003 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396116+0000", 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5160000000000002e-05 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396254+0000", 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137954 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396351+0000", 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7076000000000004e-05 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18437 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6181166439999997, 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047245167999999997, 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18437 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7199999999999999e-07 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351412+0000", 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3200000000000001e-06 2024-09-21T23:15:26.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351421+0000", 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0280000000000006e-06 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351428+0000", 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2490000000000004e-06 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392130+0000", 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040701945000000003 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392258+0000", 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012822500000000001 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392878+0000", 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062008699999999996 2024-09-21T23:15:26.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396380+0000", 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035021879999999998 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398426+0000", 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002046141 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398563+0000", 2024-09-21T23:15:26.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013656200000000001 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398652+0000", 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8850999999999993e-05 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18438 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6180820550000004, 2024-09-21T23:15:26.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047837194, 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18438 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351442+0000", 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0200000000000002e-07 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351447+0000", 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5979999999999999e-06 2024-09-21T23:15:26.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351457+0000", 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9850000000000001e-06 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351464+0000", 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394225+0000", 2024-09-21T23:15:26.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042761087000000003 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394351+0000", 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012576199999999999 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394839+0000", 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048866800000000005 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398696+0000", 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038569419999999999 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399092+0000", 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039590699999999998 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399184+0000", 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2386000000000006e-05 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399279+0000", 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4024999999999995e-05 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18754 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4255366609999998, 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040550092000000003, 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.036 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18754 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:26.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543990+0000", 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543997+0000", 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7880000000000001e-06 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544002+0000", 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578134+0000", 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.034132205999999998 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578328+0000", 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019426299999999999 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578872+0000", 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054318300000000003 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582350+0000", 2024-09-21T23:15:26.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034784690000000001 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584435+0000", 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020848820000000001 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584490+0000", 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4765999999999997e-05 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584537+0000", 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7111999999999999e-05 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18756 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4255142120000004, 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042644648, 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.042 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18756 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544010+0000", 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544013+0000", 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4910000000000001e-06 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544020+0000", 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8639999999999998e-06 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544025+0000", 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4469999999999997e-06 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580135+0000", 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036109943999999998 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580291+0000", 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015522600000000001 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580766+0000", 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047577900000000001 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586496+0000", 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057296990000000004 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586516+0000", 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0326000000000002e-05 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586576+0000", 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.965e-05 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586654+0000", 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7775000000000007e-05 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18758 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4253829869999999, 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042728264000000002, 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18758 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544141+0000", 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2099999999999995e-07 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544146+0000", 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9559999999999996e-06 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544157+0000", 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0465999999999999e-05 2024-09-21T23:15:26.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544163+0000", 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4339999999999997e-06 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582457+0000", 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.038293466999999998 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582614+0000", 2024-09-21T23:15:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015732699999999999 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.583297+0000", 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000682724 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586742+0000", 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003444964 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586755+0000", 2024-09-21T23:15:26.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3689999999999999e-05 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586802+0000", 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6841000000000001e-05 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586869+0000", 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6574000000000003e-05 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18759 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4252510110000003, 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047298266999999998, 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18759 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.054 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544273+0000", 2024-09-21T23:15:26.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4600000000000004e-07 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544278+0000", 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0579999999999997e-06 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544288+0000", 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0104e-05 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544295+0000", 2024-09-21T23:15:26.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4869999999999999e-06 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584612+0000", 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040317510000000001 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584745+0000", 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133051 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.585225+0000", 2024-09-21T23:15:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000480054 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591274+0000", 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060487429999999997 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591414+0000", 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014027700000000001 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591498+0000", 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3231000000000003e-05 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591571+0000", 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3005999999999999e-05 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18760 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:26.059 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4252192910000003, 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047443218000000002, 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18760 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544305+0000", 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544308+0000", 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3680000000000002e-06 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544315+0000", 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000003e-06 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544321+0000", 2024-09-21T23:15:26.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.507e-06 2024-09-21T23:15:26.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586945+0000", 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042624414999999999 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.587111+0000", 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016594099999999999 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.587753+0000", 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064202200000000001 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591388+0000", 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036342829999999999 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591591+0000", 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020310700000000001 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591679+0000", 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8796999999999997e-05 2024-09-21T23:15:26.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591747+0000", 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7879999999999994e-05 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18761 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:26.066 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.425192558, 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053139912999999997, 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18761 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:26.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544335+0000", 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544342+0000", 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9879999999999998e-06 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544347+0000", 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.589606+0000", 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045258537000000001 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.589780+0000", 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017489299999999999 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.590363+0000", 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058220899999999998 2024-09-21T23:15:26.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597288+0000", 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069252100000000002 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597312+0000", 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4057e-05 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597396+0000", 2024-09-21T23:15:26.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4449000000000004e-05 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597471+0000", 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.436e-05 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18762 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.425169425, 2024-09-21T23:15:26.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.05334962, 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18762 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3599999999999999e-07 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544361+0000", 2024-09-21T23:15:26.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0269999999999998e-06 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544372+0000", 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0359e-05 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544378+0000", 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8139999999999997e-06 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591881+0000", 2024-09-21T23:15:26.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047503385000000002 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.592076+0000", 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019506100000000001 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.592696+0000", 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062034199999999996 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597489+0000", 2024-09-21T23:15:26.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047924430000000004 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597556+0000", 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7407e-05 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597653+0000", 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7225000000000005e-05 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597704+0000", 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0120999999999997e-05 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18763 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4251378289999996, 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053606662999999999, 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18763 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:26.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544389+0000", 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2119999999999999e-06 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544397+0000", 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2910000000000002e-06 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544402+0000", 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.593900+0000", 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049497541999999999 2024-09-21T23:15:26.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.594001+0000", 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101384 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.594385+0000", 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038437500000000001 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597728+0000", 2024-09-21T23:15:26.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003342303 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597831+0000", 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102977 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597927+0000", 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6080000000000002e-05 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597992+0000", 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5408999999999997e-05 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18764 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.425114325, 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053769346000000003, 2024-09-21T23:15:26.084 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18764 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3300000000000003e-07 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544413+0000", 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.236e-06 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544420+0000", 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8859999999999998e-06 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544425+0000", 2024-09-21T23:15:26.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.48e-06 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.595366+0000", 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050940791999999999 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.595531+0000", 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016486100000000001 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.596215+0000", 2024-09-21T23:15:26.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068378600000000003 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598024+0000", 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001809361 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598080+0000", 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6538e-05 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598135+0000", 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4703000000000001e-05 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598178+0000", 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3269999999999997e-05 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:26.090 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:26.092 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:26.126 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-21T23:15:26.127 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:26.124+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:26.176 INFO:teuthology.orchestra.run.smithi170.stdout:1246 read 0x7bbc800 thru 0x7bc08dd (0x40de bytes) 2024-09-21T23:15:26.177 INFO:teuthology.orchestra.run.smithi170.stdout:1249 write 0xda42800 thru 0xda444c7 (0x1cc8 bytes) 2024-09-21T23:15:26.178 INFO:teuthology.orchestra.run.smithi170.stdout:1251 read 0x6787600 thru 0x678cad4 (0x54d5 bytes) 2024-09-21T23:15:26.179 INFO:teuthology.orchestra.run.smithi170.stdout:1252 punch from 0xb366800 to 0xb371e00, (0xb600 bytes) 2024-09-21T23:15:26.196 INFO:teuthology.orchestra.run.smithi170.stdout:1254 punch from 0xd94ac00 to 0xd94ae00, (0x200 bytes) 2024-09-21T23:15:26.203 INFO:teuthology.orchestra.run.smithi170.stdout:1255 write 0xb06c600 thru 0xb07283c (0x623d bytes) 2024-09-21T23:15:26.203 INFO:teuthology.orchestra.run.smithi170.stdout:1257 punch from 0xdeb6e00 to 0xdebcc00, (0x5e00 bytes) 2024-09-21T23:15:26.212 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:26.212+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:26.227 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:26.228 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:26.228 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:26.228 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:26.241 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:26.295 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:26.291+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:26.302 INFO:teuthology.orchestra.run.smithi170.stdout:1258 trunc from 0xed12600 to 0x4e4c200 2024-09-21T23:15:26.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-21T23:15:26.372 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:26.372 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:26.372 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:26.372 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:26.372 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:26.376 INFO:teuthology.orchestra.run.smithi170.stdout:1259 write 0xe232200 thru 0xe23e030 (0xbe31 bytes) 2024-09-21T23:15:26.377 INFO:teuthology.orchestra.run.smithi170.stdout:1262 punch from 0xd82a800 to 0xd839a00, (0xf200 bytes) 2024-09-21T23:15:26.378 INFO:teuthology.orchestra.run.smithi170.stdout:1263 read 0x2c14400 thru 0x2c19cd8 (0x58d9 bytes) 2024-09-21T23:15:26.379 INFO:teuthology.orchestra.run.smithi170.stdout:1264 write 0x216a600 thru 0x216db89 (0x358a bytes) 2024-09-21T23:15:26.379 INFO:teuthology.orchestra.run.smithi170.stdout:1267 punch from 0x1f0f600 to 0x1f1f400, (0xfe00 bytes) 2024-09-21T23:15:26.379 INFO:teuthology.orchestra.run.smithi170.stdout:1269 punch from 0xb9e6600 to 0xb9edc00, (0x7600 bytes) 2024-09-21T23:15:26.387 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:26.410 INFO:teuthology.orchestra.run.smithi170.stdout:1270 write 0xe378800 thru 0xe378921 (0x122 bytes) 2024-09-21T23:15:26.412 INFO:teuthology.orchestra.run.smithi170.stdout:1272 punch from 0xb4e7a00 to 0xb4e8600, (0xc00 bytes) 2024-09-21T23:15:26.416 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:26.416+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:26.470 INFO:teuthology.orchestra.run.smithi170.stdout:1273 trunc from 0xe378a00 to 0x1c9e00 2024-09-21T23:15:26.496 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:26.491+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:26.519 INFO:teuthology.orchestra.run.smithi170.stdout:1275 read 0x9be00 thru 0x9f3fa (0x35fb bytes) 2024-09-21T23:15:26.523 INFO:teuthology.orchestra.run.smithi170.stdout:1279 read 0x106400 thru 0x1076f8 (0x12f9 bytes) 2024-09-21T23:15:26.537 INFO:teuthology.orchestra.run.smithi170.stdout:1280 trunc from 0x1c9e00 to 0x627b000 2024-09-21T23:15:26.552 INFO:teuthology.orchestra.run.smithi170.stdout:1282 write 0x618bc00 thru 0x6198d66 (0xd167 bytes) 2024-09-21T23:15:26.552 INFO:teuthology.orchestra.run.smithi170.stdout:1285 read 0x1319600 thru 0x1327b1e (0xe51f bytes) 2024-09-21T23:15:26.562 INFO:teuthology.orchestra.run.smithi170.stdout:1288 trunc from 0x627b000 to 0x214ea00 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 40.341616361, 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:26.564 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:26.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.570 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.829211078, 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.571 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:26.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:26.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.576 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.829146174999998, 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.577 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:26.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.828999736, 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:26.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:26.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:26.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.588 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.828930620999998, 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:26.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:26.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.828318295999999, 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:26.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:26.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.828113891000001, 2024-09-21T23:15:26.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:26.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.828071104999999, 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.606 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:26.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:26.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.437112816999999, 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.612 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:26.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:26.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.437026306999996, 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.618 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:26.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:26.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436949417999998, 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.624 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:26.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:26.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436744578000003, 2024-09-21T23:15:26.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:26.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436656296000002, 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:26.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436525467999999, 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.645 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:26.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:26.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436479814999998, 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:26.652 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:26.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:26.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436441387999999, 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.659 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:26.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:26.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:26.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:26.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:26.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:26.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:26.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:26.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:26.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:26.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:26.668 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.669 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.669 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.669 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:26.670 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436311926000002, 2024-09-21T23:15:26.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:26.670 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.670 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.670 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:26.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.671 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:26.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:26.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:26.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:26.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:26.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:26.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:26.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:26.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:26.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:26.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:26.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:26.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:26.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:26.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:26.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:26.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:26.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:26.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:26.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:26.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:26.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:26.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:26.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:26.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.681 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.682 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.682 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:26.682 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436290284000002, 2024-09-21T23:15:26.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:26.683 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.683 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.683 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.684 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:26.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.684 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:26.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:26.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:26.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:26.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:26.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:26.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:26.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:26.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:26.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:26.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:26.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:26.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:26.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:26.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:26.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:26.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:26.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:26.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:26.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:26.694 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.695 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.695 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.695 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:26.696 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436270239999999, 2024-09-21T23:15:26.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:26.696 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.696 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.696 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.697 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:26.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.697 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:26.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:26.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:26.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:26.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:26.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:26.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:26.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:26.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:26.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:26.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:26.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:26.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:26.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:26.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:26.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:26.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:26.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:26.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:26.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:26.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:26.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:26.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:26.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:26.708 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 39.436186247999998, 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.716 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:26.719 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:26.720 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:26.711+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1289 write 0x26fc00 thru 0x270db7 (0x11b8 bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1291 read 0x911600 thru 0x915219 (0x3c1a bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1292 write 0xde94000 thru 0xde98f57 (0x4f58 bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1293 read 0x503a600 thru 0x50476cb (0xd0cc bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1296 read 0xa8e2600 thru 0xa8e79aa (0x53ab bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1298 read 0xbb38400 thru 0xbb462c5 (0xdec6 bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1299 read 0x5dfda00 thru 0x5e0788a (0x9e8b bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1300 read 0x491f200 thru 0x49205be (0x13bf bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1301 read 0xb0a3200 thru 0xb0ac603 (0x9404 bytes) 2024-09-21T23:15:26.721 INFO:teuthology.orchestra.run.smithi170.stdout:1302 write 0x25d8a00 thru 0x25e2296 (0x9897 bytes) 2024-09-21T23:15:26.722 INFO:teuthology.orchestra.run.smithi170.stdout:1303 read 0x91e0800 thru 0x91ef601 (0xee02 bytes) 2024-09-21T23:15:26.722 INFO:teuthology.orchestra.run.smithi170.stdout:1304 read 0x7065000 thru 0x70676c2 (0x26c3 bytes) 2024-09-21T23:15:26.722 INFO:teuthology.orchestra.run.smithi170.stdout:1305 write 0x7dea00 thru 0x7e5bbd (0x71be bytes) 2024-09-21T23:15:26.722 INFO:teuthology.orchestra.run.smithi170.stdout:1309 clone 5 order 23 su 4194304 sc 2 2024-09-21T23:15:26.819 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:26.815+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:26.825 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:15:26.904 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:4473 3.9 3:94174e9f:::rbd_header.10f156ded9d7:head [call rbd.parent_get in=13b out=42b,call rbd.parent_overlap_get in=29b out=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.902076+0000", 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00058712300000000005, 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00063789100000000002, 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 4473 2024-09-21T23:15:26.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902076+0000", 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902076+0000", 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902077+0000", 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902082+0000", 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2259999999999996e-06 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902096+0000", 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4392999999999999e-05 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:26.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902105+0000", 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3559999999999993e-06 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902221+0000", 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000116921 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902454+0000", 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023243100000000001 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.902714+0000", 2024-09-21T23:15:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025973700000000001 2024-09-21T23:15:26.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.909 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:26.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:26.909 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:15:26.909 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:15:26.909 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:26.919 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:26.963 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:26.959+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:26.992 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone4 from 0xde99000 (overlap 0x1c9e00) to 0x0 2024-09-21T23:15:27.020 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:27.015+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:27.094 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:27.094 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:27.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:27.121 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:27.120+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:27.222 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:27.220+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:27.278 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.953277714999999, 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076817589999999998, 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 9 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:27.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319781+0000", 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3529999999999997e-06 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319821+0000", 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0571999999999997e-05 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319871+0000", 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0114999999999997e-05 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319900+0000", 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8802999999999998e-05 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320073+0000", 2024-09-21T23:15:27.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000172669 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320090+0000", 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7223000000000001e-05 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325068+0000", 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049774889999999999 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325233+0000", 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016537600000000001 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325491+0000", 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025775700000000001 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.326967+0000", 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001476684 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327236+0000", 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026841300000000001 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327389+0000", 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0001528 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327458+0000", 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9505000000000006e-05 2024-09-21T23:15:27.285 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2008 3.3 3:c4d74490:::rbd_data.10f16a447f06.00000000000000b1:head [set-alloc-hint object_size 524288 write_size 524288,write 354304~14336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.889080105000001, 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088517769999999999, 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:27.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2008 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383977+0000", 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.959e-06 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384014+0000", 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6659999999999998e-05 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384039+0000", 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5745000000000002e-05 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384057+0000", 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7575999999999999e-05 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384162+0000", 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010501 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384169+0000", 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390619+0000", 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0064506559999999999 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390698+0000", 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8777999999999997e-05 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390782+0000", 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3552000000000002e-05 2024-09-21T23:15:27.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.391690+0000", 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000908609 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392719+0000", 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001029051 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392782+0000", 2024-09-21T23:15:27.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2644999999999994e-05 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392826+0000", 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3718999999999999e-05 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4405.0:2371 3.15 e28/25 3:afa0cd39:::rbd_data.10f16a447f06.0000000000000019:head v 28'3, mlcod=27'1)", 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.757304476000002, 2024-09-21T23:15:27.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076557819999999999, 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515751+0000", 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.547e-06 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515784+0000", 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2756000000000003e-05 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515810+0000", 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6259999999999999e-05 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515827+0000", 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6846000000000001e-05 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515987+0000", 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159729 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.516072+0000", 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4994999999999995e-05 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523367+0000", 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0072950250000000001 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523405+0000", 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8624000000000002e-05 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.296 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6155 3.1a 3:58809d19:::rbd_data.10f1e443a6fa.000000000000005f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.779392253999999, 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0075748489999999998, 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6155 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:27.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493664+0000", 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0470000000000001e-06 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493674+0000", 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7659999999999993e-06 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493696+0000", 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717e-05 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493717+0000", 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1002e-05 2024-09-21T23:15:27.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495059+0000", 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0013413570000000001 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495322+0000", 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026359099999999998 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495706+0000", 2024-09-21T23:15:27.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038393999999999999 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.496952+0000", 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001245833 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.500994+0000", 2024-09-21T23:15:27.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0040416849999999997 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.501122+0000", 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128541 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.501237+0000", 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011437 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:253 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311625+0000", 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.96142934899999999, 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.008580786, 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 253 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311625+0000", 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311625+0000", 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311626+0000", 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9899999999999999e-07 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311630+0000", 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1439999999999996e-06 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311639+0000", 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3279999999999995e-06 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311648+0000", 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.234e-06 2024-09-21T23:15:27.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.313995+0000", 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023464060000000001 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.314065+0000", 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0815000000000006e-05 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320048+0000", 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059821359999999999 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320110+0000", 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2062000000000003e-05 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320206+0000", 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5861999999999998e-05 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.307 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:254 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.96139232100000005, 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.008689034, 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 254 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1500000000000004e-07 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311667+0000", 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5109999999999997e-06 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311677+0000", 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8730000000000002e-06 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311683+0000", 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8560000000000003e-06 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.314885+0000", 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0032020389999999998 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.314935+0000", 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0683000000000002e-05 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320215+0000", 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0052797870000000002 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320278+0000", 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3142e-05 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320351+0000", 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2527999999999994e-05 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:255 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:27.312 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.96135604399999997, 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088296720000000002, 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 255 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311701+0000", 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9380000000000001e-06 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311712+0000", 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0421999999999999e-05 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311719+0000", 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4730000000000003e-06 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315379+0000", 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00365936 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315427+0000", 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8838000000000002e-05 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320372+0000", 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049444349999999996 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320474+0000", 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101902 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320528+0000", 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3940999999999997e-05 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:256 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.317 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.96132163800000003, 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088795980000000007, 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 256 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311733+0000", 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311738+0000", 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.989e-06 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311748+0000", 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5370000000000003e-06 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311754+0000", 2024-09-21T23:15:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315953+0000", 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0041992679999999999 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.316028+0000", 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5337000000000001e-05 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320534+0000", 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045061620000000002 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320567+0000", 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2251e-05 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320612+0000", 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5425e-05 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.322 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:347 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.376936+0000", 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.89611836700000003, 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088040630000000009, 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 347 2024-09-21T23:15:27.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376936+0000", 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376936+0000", 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376937+0000", 2024-09-21T23:15:27.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1739999999999999e-06 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376943+0000", 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1800000000000001e-06 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376969+0000", 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5681999999999999e-05 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376981+0000", 2024-09-21T23:15:27.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2177999999999999e-05 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377118+0000", 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013718500000000001 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377278+0000", 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159531 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385613+0000", 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.008335716 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385669+0000", 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5937999999999999e-05 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385740+0000", 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0479000000000001e-05 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:348 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.377244+0000", 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.89581030399999995, 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0086141900000000007, 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 348 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377244+0000", 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377244+0000", 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377245+0000", 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1900000000000001e-07 2024-09-21T23:15:27.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377250+0000", 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5470000000000004e-06 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377269+0000", 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9208e-05 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377277+0000", 2024-09-21T23:15:27.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4030000000000001e-06 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378901+0000", 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016243080000000001 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379185+0000", 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028430300000000002 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385749+0000", 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065638579999999997 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385805+0000", 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5476999999999998e-05 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385858+0000", 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3167000000000001e-05 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:349 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.89575860500000004, 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0093055059999999998, 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 349 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.333 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377296+0000", 2024-09-21T23:15:27.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3499999999999996e-07 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377301+0000", 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4270000000000001e-06 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377309+0000", 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3899999999999993e-06 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377315+0000", 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4200000000000004e-06 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.380850+0000", 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035341690000000002 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.380984+0000", 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134857 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386492+0000", 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055073939999999997 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386540+0000", 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8424999999999998e-05 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386601+0000", 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0788999999999997e-05 2024-09-21T23:15:27.337 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:353 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378238+0000", 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.89481575899999999, 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084567879999999998, 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 353 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378238+0000", 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378238+0000", 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378239+0000", 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1149999999999999e-06 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378245+0000", 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378267+0000", 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1454000000000002e-05 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378277+0000", 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9280000000000005e-06 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.382750+0000", 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0044729230000000002 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.382995+0000", 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024587600000000002 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386610+0000", 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036147169999999999 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386639+0000", 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8924999999999998e-05 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386695+0000", 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6023999999999998e-05 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.342 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:354 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.89475548299999996, 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096360100000000004, 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 354 2024-09-21T23:15:27.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8999999999999996e-07 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378304+0000", 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4479999999999996e-06 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378314+0000", 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.028e-05 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378321+0000", 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6340000000000003e-06 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.384551+0000", 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0062308509999999999 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.384845+0000", 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029336299999999999 2024-09-21T23:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387768+0000", 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0029234249999999999 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387860+0000", 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2260000000000001e-05 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387935+0000", 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4159000000000006e-05 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:355 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.894576233, 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010205018999999999, 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.348 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 355 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378479+0000", 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3600000000000003e-07 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378484+0000", 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3680000000000001e-06 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378497+0000", 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3086e-05 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378504+0000", 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2509999999999997e-06 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385893+0000", 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073889749999999999 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385942+0000", 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9002999999999998e-05 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388509+0000", 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002567036 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388601+0000", 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000001e-05 2024-09-21T23:15:27.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388683+0000", 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2293999999999995e-05 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:356 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.89386018599999995, 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095977030000000008, 2024-09-21T23:15:27.353 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 356 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379195+0000", 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7400000000000002e-07 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379200+0000", 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3229999999999997e-06 2024-09-21T23:15:27.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379212+0000", 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1677e-05 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379219+0000", 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4959999999999999e-06 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386720+0000", 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.007500657 2024-09-21T23:15:27.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386785+0000", 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4667999999999998e-05 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388690+0000", 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001905452 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388726+0000", 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5565999999999998e-05 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388792+0000", 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5989999999999997e-05 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:911 3.1a 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997360+0000", 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.27569374800000002, 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074815200000000002, 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.358 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 911 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997360+0000", 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997360+0000", 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997362+0000", 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997369+0000", 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.396e-06 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997395+0000", 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5372e-05 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997407+0000", 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2404e-05 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997676+0000", 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026939599999999998 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997807+0000", 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013108999999999999 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998111+0000", 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030347399999999997 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.002558+0000", 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00444707 2024-09-21T23:15:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.004696+0000", 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0021381239999999999 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.004756+0000", 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0106999999999999e-05 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.004842+0000", 2024-09-21T23:15:27.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5654000000000004e-05 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:914 3.1 3:873c0de2:::rbd_data.10f192b3a1cb.0000000000000007:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997523+0000", 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.27553082499999998, 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076758390000000003, 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 914 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997523+0000", 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997523+0000", 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997524+0000", 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.92e-07 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997528+0000", 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1030000000000004e-06 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997539+0000", 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997548+0000", 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8810000000000002e-06 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998685+0000", 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011365699999999999 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.999466+0000", 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078138100000000005 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005057+0000", 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055907960000000003 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005124+0000", 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6798999999999995e-05 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005199+0000", 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5372999999999998e-05 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.368 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:916 3.a 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997573+0000", 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.27548107100000002, 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0086410089999999998, 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 916 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.369 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997573+0000", 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997573+0000", 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997574+0000", 2024-09-21T23:15:27.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997578+0000", 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4850000000000003e-06 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997589+0000", 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1066e-05 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997597+0000", 2024-09-21T23:15:27.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998872+0000", 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012746159999999999 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998971+0000", 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8659000000000005e-05 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:27.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.999958+0000", 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098702600000000001 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005289+0000", 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005331438 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.006069+0000", 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078003899999999995 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.006155+0000", 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5934999999999996e-05 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.006214+0000", 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9072e-05 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:922 3.3 3:c62ba2ad:::rbd_data.10f192b3a1cb.000000000000001b:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:27.374 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997824+0000", 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.27523033499999999, 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074398679999999997, 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 922 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997824+0000", 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997824+0000", 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997825+0000", 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1519999999999999e-06 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997831+0000", 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2169999999999999e-06 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997847+0000", 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5461e-05 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997856+0000", 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5100000000000004e-06 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997982+0000", 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000125671 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998077+0000", 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4866999999999998e-05 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005121+0000", 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070439400000000003 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005178+0000", 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7357e-05 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005264+0000", 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5692999999999994e-05 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:928 3.14 3:2ad4dd57:::rbd_data.10f192b3a1cb.0000000000000093:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.998365+0000", 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.274689189, 2024-09-21T23:15:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073553120000000001, 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 928 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998365+0000", 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998365+0000", 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998366+0000", 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9500000000000002e-07 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998369+0000", 2024-09-21T23:15:27.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8099999999999999e-06 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998379+0000", 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0158e-05 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998387+0000", 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0450000000000003e-06 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.000669+0000", 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002281993 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.000828+0000", 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015918900000000001 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.001151+0000", 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032344300000000003 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005448+0000", 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0042969169999999999 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005553+0000", 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105194 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005654+0000", 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010076100000000001 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005720+0000", 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6206999999999998e-05 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.385 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:27.386 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:27.387 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:27.388 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:27.383+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:27.488 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:27.488+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:27.522 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:27.541 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:27.541 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:27.559 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:27.559 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:27.567 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4662.0:59 3.1d 3:bd4ac07f:::rbd_data.10f156ded9d7.0000000000000074:head [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:27.566811+0000", 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00031059700000000002, 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035576100000000001, 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:27.569 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4662", 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/434974820", 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 59 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.566811+0000", 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.566811+0000", 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:27.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.566813+0000", 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.544e-06 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.566821+0000", 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4260000000000003e-06 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.566850+0000", 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8855000000000001e-05 2024-09-21T23:15:27.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.566864+0000", 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4216000000000001e-05 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.566965+0000", 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100542 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.567084+0000", 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011935299999999999 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.572 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:27.573 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.573 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:27.573 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:15:27.573 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:15:27.573 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:27.580 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:27.580 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:27.583 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:27.599 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:27.599 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:27.617 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:27.617 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:27.622 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:27.622 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:27.631 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:27.627+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:27.689 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:27.683+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:27.710 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:27.710 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:27.726 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:27.811 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:27.807+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:27.891 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:27.887+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:27.895 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:27.895 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:27.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:27.992 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:27.987+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:28.051 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:28.051 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:28.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:28.093 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:28.087+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:28.193 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:28.193 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:28.194 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:28.192+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:28.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:28.295 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:28.291+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:28.377 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:28.377 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:28.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:28.436 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:28.436+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:28.497 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:28.491+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:28.513 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:28.514 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:28.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:28.598 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:28.591+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:28.699 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:28.696+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:28.704 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:28.704 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:28.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:28.804 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:28.804+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:28.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:15:28.886 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:28.886 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:28.900 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:28.900+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:28.901 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:29.001 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:28.995+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:29.033 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:29.033 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:29.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:29.107 INFO:teuthology.orchestra.run.smithi170.stdout:1310 read 0x1edd800 thru 0x1eec941 (0xf142 bytes) 2024-09-21T23:15:29.108 INFO:teuthology.orchestra.run.smithi170.stdout:1312 punch from 0x3a9fa00 to 0x3aa8200, (0x8800 bytes) 2024-09-21T23:15:29.121 INFO:teuthology.orchestra.run.smithi170.stdout:1314 read 0x9b63800 thru 0x9b6e395 (0xab96 bytes) 2024-09-21T23:15:29.121 INFO:teuthology.orchestra.run.smithi170.stdout:1315 write 0xd382c00 thru 0xd388b0e (0x5f0f bytes) 2024-09-21T23:15:29.122 INFO:teuthology.orchestra.run.smithi170.stdout:1316 read 0x48d1200 thru 0x48d893c (0x773d bytes) 2024-09-21T23:15:29.135 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:29.131+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:29.151 INFO:teuthology.orchestra.run.smithi170.stdout:1317 write 0xe537200 thru 0xe53deb1 (0x6cb2 bytes) 2024-09-21T23:15:29.203 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:29.199+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:29.219 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:29.219 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:29.231 INFO:teuthology.orchestra.run.smithi170.stdout:1319 trunc from 0xe53e000 to 0x27f8600 2024-09-21T23:15:29.233 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:29.285 INFO:teuthology.orchestra.run.smithi170.stdout:1324 punch from 0x9bba00 to 0x9c4e00, (0x9400 bytes) 2024-09-21T23:15:29.287 INFO:teuthology.orchestra.run.smithi170.stdout:1326 read 0x1b9e600 thru 0x1baa0c5 (0xbac6 bytes) 2024-09-21T23:15:29.319 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:29.315+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:29.333 INFO:teuthology.orchestra.run.smithi170.stdout:1327 write 0xbe03a00 thru 0xbe0dd91 (0xa392 bytes) 2024-09-21T23:15:29.336 INFO:teuthology.orchestra.run.smithi170.stdout:1333 read 0x5ca8200 thru 0x5ca8d30 (0xb31 bytes) 2024-09-21T23:15:29.336 INFO:teuthology.orchestra.run.smithi170.stdout:1334 read 0x739f200 thru 0x73a3a3c (0x483d bytes) 2024-09-21T23:15:29.336 INFO:teuthology.orchestra.run.smithi170.stdout:1335 write 0x1ba8200 thru 0x1bb7c1d (0xfa1e bytes) 2024-09-21T23:15:29.338 INFO:teuthology.orchestra.run.smithi170.stdout:1336 trunc from 0xbe0de00 to 0xceeee00 2024-09-21T23:15:29.375 INFO:teuthology.orchestra.run.smithi170.stdout:1340 write 0x6c3e800 thru 0x6c4221c (0x3a1d bytes) 2024-09-21T23:15:29.398 INFO:teuthology.orchestra.run.smithi170.stdout:1341 trunc from 0xceeee00 to 0x52e1800 2024-09-21T23:15:29.398 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:29.398 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:29.405 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:29.404+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:29.413 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:29.479 INFO:teuthology.orchestra.run.smithi170.stdout:1342 write 0x9a0c400 thru 0x9a16cd5 (0xa8d6 bytes) 2024-09-21T23:15:29.481 INFO:teuthology.orchestra.run.smithi170.stdout:1344 write 0x53ac200 thru 0x53b3850 (0x7651 bytes) 2024-09-21T23:15:29.490 INFO:teuthology.orchestra.run.smithi170.stdout:1346 trunc from 0x9a16e00 to 0xe254600 2024-09-21T23:15:29.506 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:29.504+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:29.517 INFO:teuthology.orchestra.run.smithi170.stdout:1350 punch from 0xb112200 to 0xb117000, (0x4e00 bytes) 2024-09-21T23:15:29.518 INFO:teuthology.orchestra.run.smithi170.stdout:1351 punch from 0x3e1c200 to 0x3e24200, (0x8000 bytes) 2024-09-21T23:15:29.518 INFO:teuthology.orchestra.run.smithi170.stdout:1352 read 0x9d5c800 thru 0x9d63682 (0x6e83 bytes) 2024-09-21T23:15:29.529 INFO:teuthology.orchestra.run.smithi170.stdout:1355 trunc from 0xe254600 to 0x9a17600 2024-09-21T23:15:29.563 INFO:teuthology.orchestra.run.smithi170.stdout:1357 punch from 0x44e1400 to 0x44efa00, (0xe600 bytes) 2024-09-21T23:15:29.583 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:29.583 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:29.598 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:29.600 INFO:teuthology.orchestra.run.smithi170.stdout:1358 write 0xdc54800 thru 0xdc563e4 (0x1be5 bytes) 2024-09-21T23:15:29.602 INFO:teuthology.orchestra.run.smithi170.stdout:1359 read 0x7be9000 thru 0x7be9e47 (0xe48 bytes) 2024-09-21T23:15:29.602 INFO:teuthology.orchestra.run.smithi170.stdout:1360 write 0x6a89400 thru 0x6a94f51 (0xbb52 bytes) 2024-09-21T23:15:29.602 INFO:teuthology.orchestra.run.smithi170.stdout:1362 read 0x52d0a00 thru 0x52da408 (0x9a09 bytes) 2024-09-21T23:15:29.603 INFO:teuthology.orchestra.run.smithi170.stdout:1365 write 0xd22fe00 thru 0xd234606 (0x4807 bytes) 2024-09-21T23:15:29.624 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:29.625 INFO:teuthology.orchestra.run.smithi170.stdout:1366 trunc from 0xdc56400 to 0x4b87800 2024-09-21T23:15:29.643 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:29.639+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:29.675 INFO:teuthology.orchestra.run.smithi170.stdout:1368 trunc from 0x4b87800 to 0xa23b400 2024-09-21T23:15:29.691 INFO:teuthology.orchestra.run.smithi170.stdout:1369 read 0x2cd6a00 thru 0x2cdaae2 (0x40e3 bytes) 2024-09-21T23:15:29.707 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:29.704+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:29.725 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:29.725 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:29.733 INFO:teuthology.orchestra.run.smithi170.stdout:1372 trunc from 0xa23b400 to 0x1253600 2024-09-21T23:15:29.740 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:29.763 INFO:teuthology.orchestra.run.smithi170.stdout:1373 read 0x37b200 thru 0x381679 (0x647a bytes) 2024-09-21T23:15:29.774 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:29.791 INFO:teuthology.orchestra.run.smithi170.stdout:1376 trunc from 0x1253600 to 0xcdd3400 2024-09-21T23:15:29.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:29.805 INFO:teuthology.orchestra.run.smithi170.stdout:1377 punch from 0x70c8e00 to 0x70d6800, (0xda00 bytes) 2024-09-21T23:15:29.807 INFO:teuthology.orchestra.run.smithi170.stdout:1378 read 0x7d43e00 thru 0x7d461b5 (0x23b6 bytes) 2024-09-21T23:15:29.808 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:29.804+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:29.808 INFO:teuthology.orchestra.run.smithi170.stdout:1382 write 0x824e800 thru 0x8254afe (0x62ff bytes) 2024-09-21T23:15:29.808 INFO:teuthology.orchestra.run.smithi170.stdout:1383 punch from 0x5deb200 to 0x5df0600, (0x5400 bytes) 2024-09-21T23:15:29.808 INFO:teuthology.orchestra.run.smithi170.stdout:1387 write 0x75fc200 thru 0x7601f4c (0x5d4d bytes) 2024-09-21T23:15:29.808 INFO:teuthology.orchestra.run.smithi170.stdout:1390 read 0xbe3e600 thru 0xbe4c295 (0xdc96 bytes) 2024-09-21T23:15:29.814 INFO:teuthology.orchestra.run.smithi170.stdout:1391 trunc from 0xcdd3400 to 0x9ed4200 2024-09-21T23:15:29.814 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:29.815 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:29.830 INFO:teuthology.orchestra.run.smithi170.stdout:1394 read 0x20afa00 thru 0x20ba035 (0xa636 bytes) 2024-09-21T23:15:29.831 INFO:teuthology.orchestra.run.smithi170.stdout:1395 read 0x1821200 thru 0x182f356 (0xe157 bytes) 2024-09-21T23:15:29.831 INFO:teuthology.orchestra.run.smithi170.stdout:1396 write 0x17ae800 thru 0x17b71fb (0x89fc bytes) 2024-09-21T23:15:29.831 INFO:teuthology.orchestra.run.smithi170.stdout:1398 read 0xc59200 thru 0xc63c5f (0xaa60 bytes) 2024-09-21T23:15:29.837 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:29.837 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:29.847 INFO:teuthology.orchestra.run.smithi170.stdout:1399 trunc from 0x9ed4200 to 0xdcde00 2024-09-21T23:15:29.856 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:29.856 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:29.877 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:29.877 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:29.880 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:29.880 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:29.895 INFO:teuthology.orchestra.run.smithi170.stdout:1405 read 0x44c400 thru 0x454600 (0x8201 bytes) 2024-09-21T23:15:29.904 INFO:teuthology.orchestra.run.smithi170.stdout:1408 trunc from 0xdcde00 to 0x340d600 2024-09-21T23:15:29.913 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:15:29.914 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:29.912+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:29.915 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:29.915 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:29.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:29.915 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.591955183, 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076817589999999998, 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 9 2024-09-21T23:15:29.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319776+0000", 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319781+0000", 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3529999999999997e-06 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319821+0000", 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0571999999999997e-05 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319871+0000", 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0114999999999997e-05 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.319900+0000", 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8802999999999998e-05 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320073+0000", 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000172669 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.320090+0000", 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7223000000000001e-05 2024-09-21T23:15:29.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325068+0000", 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049774889999999999 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325233+0000", 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016537600000000001 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.325491+0000", 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025775700000000001 2024-09-21T23:15:29.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.326967+0000", 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001476684 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327236+0000", 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026841300000000001 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327389+0000", 2024-09-21T23:15:29.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0001528 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:03.327458+0000", 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9505000000000006e-05 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2008 3.3 3:c4d74490:::rbd_data.10f16a447f06.00000000000000b1:head [set-alloc-hint object_size 524288 write_size 524288,write 354304~14336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:29.922 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.527757572999999, 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088517769999999999, 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2008 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383977+0000", 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.959e-06 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384014+0000", 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6659999999999998e-05 2024-09-21T23:15:29.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384039+0000", 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5745000000000002e-05 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384057+0000", 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7575999999999999e-05 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384162+0000", 2024-09-21T23:15:29.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010501 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384169+0000", 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390619+0000", 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0064506559999999999 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390698+0000", 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8777999999999997e-05 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390782+0000", 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3552000000000002e-05 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.391690+0000", 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000908609 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392719+0000", 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001029051 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392782+0000", 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2644999999999994e-05 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392826+0000", 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3718999999999999e-05 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4405.0:2371 3.15 e28/25 3:afa0cd39:::rbd_data.10f16a447f06.0000000000000019:head v 28'3, mlcod=27'1)", 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.395981943999999, 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076557819999999999, 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.929 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515750+0000", 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515751+0000", 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.547e-06 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515784+0000", 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2756000000000003e-05 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515810+0000", 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6259999999999999e-05 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515827+0000", 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6846000000000001e-05 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.515987+0000", 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159729 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.516072+0000", 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4994999999999995e-05 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523367+0000", 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0072950250000000001 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523405+0000", 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8624000000000002e-05 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4506.0:6155 3.1a 3:58809d19:::rbd_data.10f1e443a6fa.000000000000005f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.418069722, 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0075748489999999998, 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4506", 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 6155 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:29.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493662+0000", 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493664+0000", 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0470000000000001e-06 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493674+0000", 2024-09-21T23:15:29.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7659999999999993e-06 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493696+0000", 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717e-05 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.493717+0000", 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1002e-05 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495059+0000", 2024-09-21T23:15:29.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0013413570000000001 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495322+0000", 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026359099999999998 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.495706+0000", 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038393999999999999 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.496952+0000", 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001245833 2024-09-21T23:15:29.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.500994+0000", 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0040416849999999997 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.501122+0000", 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128541 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:16.501237+0000", 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011437 2024-09-21T23:15:29.938 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:253 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311625+0000", 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.6001068169999999, 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.008580786, 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.939 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 253 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311625+0000", 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311625+0000", 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311626+0000", 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9899999999999999e-07 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311630+0000", 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1439999999999996e-06 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311639+0000", 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3279999999999995e-06 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311648+0000", 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.234e-06 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.313995+0000", 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023464060000000001 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.314065+0000", 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0815000000000006e-05 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320048+0000", 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059821359999999999 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320110+0000", 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2062000000000003e-05 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320206+0000", 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5861999999999998e-05 2024-09-21T23:15:29.943 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:254 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.600069789, 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.008689034, 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 254 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311662+0000", 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1500000000000004e-07 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311667+0000", 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5109999999999997e-06 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311677+0000", 2024-09-21T23:15:29.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8730000000000002e-06 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311683+0000", 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8560000000000003e-06 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.314885+0000", 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0032020389999999998 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.314935+0000", 2024-09-21T23:15:29.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0683000000000002e-05 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320215+0000", 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0052797870000000002 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320278+0000", 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3142e-05 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320351+0000", 2024-09-21T23:15:29.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2527999999999994e-05 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:255 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.600033512, 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088296720000000002, 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 255 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311701+0000", 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9380000000000001e-06 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311712+0000", 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0421999999999999e-05 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311719+0000", 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4730000000000003e-06 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315379+0000", 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00365936 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315427+0000", 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8838000000000002e-05 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320372+0000", 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049444349999999996 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320474+0000", 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101902 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320528+0000", 2024-09-21T23:15:29.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3940999999999997e-05 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:256 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5999991059999998, 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088795980000000007, 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 256 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311733+0000", 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311738+0000", 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.989e-06 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311748+0000", 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5370000000000003e-06 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311754+0000", 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315953+0000", 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0041992679999999999 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.316028+0000", 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5337000000000001e-05 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320534+0000", 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045061620000000002 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320567+0000", 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2251e-05 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320612+0000", 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5425e-05 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.958 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:347 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.376936+0000", 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5347958350000002, 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088040630000000009, 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 347 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376936+0000", 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376936+0000", 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376937+0000", 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1739999999999999e-06 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376943+0000", 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1800000000000001e-06 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376969+0000", 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5681999999999999e-05 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.376981+0000", 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2177999999999999e-05 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377118+0000", 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013718500000000001 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377278+0000", 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159531 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385613+0000", 2024-09-21T23:15:29.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.008335716 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385669+0000", 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5937999999999999e-05 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385740+0000", 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0479000000000001e-05 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:348 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.377244+0000", 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5344877719999999, 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0086141900000000007, 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 348 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.964 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377244+0000", 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377244+0000", 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377245+0000", 2024-09-21T23:15:29.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1900000000000001e-07 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377250+0000", 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5470000000000004e-06 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377269+0000", 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9208e-05 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377277+0000", 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4030000000000001e-06 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378901+0000", 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016243080000000001 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379185+0000", 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028430300000000002 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385749+0000", 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065638579999999997 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385805+0000", 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5476999999999998e-05 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385858+0000", 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3167000000000001e-05 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:349 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5344360730000002, 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0093055059999999998, 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 349 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377296+0000", 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3499999999999996e-07 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377301+0000", 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4270000000000001e-06 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377309+0000", 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3899999999999993e-06 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377315+0000", 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4200000000000004e-06 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.380850+0000", 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035341690000000002 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.380984+0000", 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134857 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386492+0000", 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055073939999999997 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386540+0000", 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8424999999999998e-05 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386601+0000", 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0788999999999997e-05 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:353 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378238+0000", 2024-09-21T23:15:29.973 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5334932270000001, 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084567879999999998, 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 353 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378238+0000", 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378238+0000", 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378239+0000", 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1149999999999999e-06 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378245+0000", 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:15:29.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378267+0000", 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1454000000000002e-05 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378277+0000", 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9280000000000005e-06 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.382750+0000", 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0044729230000000002 2024-09-21T23:15:29.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.382995+0000", 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024587600000000002 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386610+0000", 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036147169999999999 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386639+0000", 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8924999999999998e-05 2024-09-21T23:15:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386695+0000", 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6023999999999998e-05 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:354 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.533432951, 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096360100000000004, 2024-09-21T23:15:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 354 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8999999999999996e-07 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378304+0000", 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4479999999999996e-06 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378314+0000", 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.028e-05 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378321+0000", 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6340000000000003e-06 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.384551+0000", 2024-09-21T23:15:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0062308509999999999 2024-09-21T23:15:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.384845+0000", 2024-09-21T23:15:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029336299999999999 2024-09-21T23:15:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387768+0000", 2024-09-21T23:15:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0029234249999999999 2024-09-21T23:15:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387860+0000", 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2260000000000001e-05 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387935+0000", 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4159000000000006e-05 2024-09-21T23:15:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:355 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.533253701, 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010205018999999999, 2024-09-21T23:15:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 355 2024-09-21T23:15:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378479+0000", 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3600000000000003e-07 2024-09-21T23:15:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378484+0000", 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3680000000000001e-06 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378497+0000", 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3086e-05 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378504+0000", 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2509999999999997e-06 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385893+0000", 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073889749999999999 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385942+0000", 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9002999999999998e-05 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388509+0000", 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002567036 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388601+0000", 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000001e-05 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388683+0000", 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2293999999999995e-05 2024-09-21T23:15:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:356 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.532537654, 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095977030000000008, 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 356 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379195+0000", 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7400000000000002e-07 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379200+0000", 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3229999999999997e-06 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379212+0000", 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1677e-05 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379219+0000", 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4959999999999999e-06 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386720+0000", 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.007500657 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386785+0000", 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4667999999999998e-05 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388690+0000", 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001905452 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388726+0000", 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5565999999999998e-05 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388792+0000", 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5989999999999997e-05 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:911 3.1a 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997360+0000", 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.9143712160000002, 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074815200000000002, 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 911 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997360+0000", 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997360+0000", 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997362+0000", 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4330000000000001e-06 2024-09-21T23:15:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997369+0000", 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.396e-06 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997395+0000", 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5372e-05 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997407+0000", 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2404e-05 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997676+0000", 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026939599999999998 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997807+0000", 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013108999999999999 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998111+0000", 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030347399999999997 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.002558+0000", 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00444707 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.004696+0000", 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0021381239999999999 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.004756+0000", 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0106999999999999e-05 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.004842+0000", 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5654000000000004e-05 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:914 3.1 3:873c0de2:::rbd_data.10f192b3a1cb.0000000000000007:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997523+0000", 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.9142082929999997, 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0076758390000000003, 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 914 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997523+0000", 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997523+0000", 2024-09-21T23:15:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997524+0000", 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.92e-07 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997528+0000", 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1030000000000004e-06 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997539+0000", 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:15:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997548+0000", 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8810000000000002e-06 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998685+0000", 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011365699999999999 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.999466+0000", 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078138100000000005 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005057+0000", 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055907960000000003 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005124+0000", 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6798999999999995e-05 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005199+0000", 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5372999999999998e-05 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:916 3.a 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997573+0000", 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.9141585389999998, 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0086410089999999998, 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 916 2024-09-21T23:15:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997573+0000", 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997573+0000", 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997574+0000", 2024-09-21T23:15:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9299999999999997e-07 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997578+0000", 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4850000000000003e-06 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997589+0000", 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1066e-05 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997597+0000", 2024-09-21T23:15:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8800000000000008e-06 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998872+0000", 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012746159999999999 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998971+0000", 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8659000000000005e-05 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.999958+0000", 2024-09-21T23:15:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098702600000000001 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005289+0000", 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005331438 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.006069+0000", 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078003899999999995 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.006155+0000", 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5934999999999996e-05 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.006214+0000", 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9072e-05 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:922 3.3 3:c62ba2ad:::rbd_data.10f192b3a1cb.000000000000001b:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.997824+0000", 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.9139078029999999, 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074398679999999997, 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 922 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997824+0000", 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997824+0000", 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997825+0000", 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1519999999999999e-06 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997831+0000", 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2169999999999999e-06 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997847+0000", 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5461e-05 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997856+0000", 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5100000000000004e-06 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.997982+0000", 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000125671 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998077+0000", 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4866999999999998e-05 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005121+0000", 2024-09-21T23:15:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070439400000000003 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005178+0000", 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7357e-05 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005264+0000", 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5692999999999994e-05 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:928 3.14 3:2ad4dd57:::rbd_data.10f192b3a1cb.0000000000000093:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.998365+0000", 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.9133666570000001, 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073553120000000001, 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 928 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998365+0000", 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998365+0000", 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998366+0000", 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9500000000000002e-07 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998369+0000", 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8099999999999999e-06 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998379+0000", 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0158e-05 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.998387+0000", 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0450000000000003e-06 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.000669+0000", 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002281993 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.000828+0000", 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015918900000000001 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.001151+0000", 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032344300000000003 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005448+0000", 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0042969169999999999 2024-09-21T23:15:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005553+0000", 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105194 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005654+0000", 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010076100000000001 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:27.005720+0000", 2024-09-21T23:15:30.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6206999999999998e-05 2024-09-21T23:15:30.024 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.024 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.024 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.024 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.024 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.024 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:30.027 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:30.024+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:30.027 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:30.126 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:30.128+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:30.152 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:30.153 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:30.153 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:30.153 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:30.166 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:30.172 INFO:teuthology.orchestra.run.smithi170.stdout:1410 read 0xb92600 thru 0xb9b9ae (0x93af bytes) 2024-09-21T23:15:30.174 INFO:teuthology.orchestra.run.smithi170.stdout:1414 punch from 0x21bf400 to 0x21c9200, (0x9e00 bytes) 2024-09-21T23:15:30.174 INFO:teuthology.orchestra.run.smithi170.stdout:1415 punch from 0x71a600 to 0x71be00, (0x1800 bytes) 2024-09-21T23:15:30.175 INFO:teuthology.orchestra.run.smithi170.stdout:1417 read 0xdd8600 thru 0xddd011 (0x4a12 bytes) 2024-09-21T23:15:30.178 INFO:teuthology.orchestra.run.smithi170.stdout:1418 read 0x1566200 thru 0x1569b9a (0x399b bytes) 2024-09-21T23:15:30.221 INFO:teuthology.orchestra.run.smithi170.stdout:1422 write 0x9d7b000 thru 0x9d8abd8 (0xfbd9 bytes) 2024-09-21T23:15:30.227 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:30.223+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:30.228 INFO:teuthology.orchestra.run.smithi170.stdout:1424 trunc from 0x9d8ac00 to 0xb8c5000 2024-09-21T23:15:30.256 INFO:teuthology.orchestra.run.smithi170.stdout:1425 read 0xb203200 thru 0xb206a04 (0x3805 bytes) 2024-09-21T23:15:30.272 INFO:teuthology.orchestra.run.smithi170.stdout:1427 trunc from 0xb8c5000 to 0x68e3c00 2024-09-21T23:15:30.291 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:30.291 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:30.291 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:30.291 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:30.291 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:30.306 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:30.316 INFO:teuthology.orchestra.run.smithi170.stdout:1430 trunc from 0x68e3c00 to 0xc032600 2024-09-21T23:15:30.328 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:30.328+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:30.341 INFO:teuthology.orchestra.run.smithi170.stdout:1432 read 0x63bf600 thru 0x63ca963 (0xb364 bytes) 2024-09-21T23:15:30.343 INFO:teuthology.orchestra.run.smithi170.stdout:1434 punch from 0xfb9000 to 0xfc6800, (0xd800 bytes) 2024-09-21T23:15:30.380 INFO:teuthology.orchestra.run.smithi170.stdout:1436 trunc from 0xc032600 to 0x2e400 2024-09-21T23:15:30.430 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:30.428+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:30.469 INFO:teuthology.orchestra.run.smithi170.stdout:1438 write 0xb85b000 thru 0xb861c1d (0x6c1e bytes) 2024-09-21T23:15:30.471 INFO:teuthology.orchestra.run.smithi170.stdout:1444 punch from 0xadb0c00 to 0xadb3c00, (0x3000 bytes) 2024-09-21T23:15:30.477 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:276 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.522817700000001, 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014002473, 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.478 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 276 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949282+0000", 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.26e-06 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949286+0000", 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9230000000000004e-06 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949289+0000", 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963078+0000", 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013788804999999999 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963169+0000", 2024-09-21T23:15:30.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1114999999999998e-05 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963283+0000", 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011409 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:277 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.522805215999998, 2024-09-21T23:15:30.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014347459999999999, 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 277 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949294+0000", 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949295+0000", 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2580000000000001e-06 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949299+0000", 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949302+0000", 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9859999999999999e-06 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963339+0000", 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014037304 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963466+0000", 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012664600000000001 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963641+0000", 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017508399999999999 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:278 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:30.486 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.522792823, 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014599439000000001, 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 278 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949307+0000", 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.139e-06 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949311+0000", 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0230000000000003e-06 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949314+0000", 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963697+0000", 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014382544000000001 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963785+0000", 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7587e-05 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963905+0000", 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012056 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:279 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.522779970999999, 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014999721000000001, 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 279 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949319+0000", 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0600000000000001e-07 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949320+0000", 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.313e-06 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949324+0000", 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9570000000000004e-06 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949327+0000", 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963958+0000", 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014630341 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964127+0000", 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016937300000000001 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964318+0000", 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019130699999999999 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:280 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.522746135999999, 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015213915, 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 280 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949353+0000", 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949361+0000", 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7709999999999995e-06 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949367+0000", 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.003e-06 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949371+0000", 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964367+0000", 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014995746000000001 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964460+0000", 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3565000000000003e-05 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964566+0000", 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106067 2024-09-21T23:15:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:350 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.505371858, 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013944434, 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.499 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 350 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.65e-07 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966728+0000", 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3069999999999999e-06 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966732+0000", 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9040000000000002e-06 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966735+0000", 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7819999999999998e-06 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980479+0000", 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013744528000000001 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980562+0000", 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2218999999999993e-05 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980671+0000", 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109429 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:351 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.505359755000001, 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014095582000000001, 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 351 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9799999999999999e-07 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966740+0000", 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966744+0000", 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:15:30.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966747+0000", 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.117e-06 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980710+0000", 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013963074000000001 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980762+0000", 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1779e-05 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980834+0000", 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2538000000000002e-05 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.507 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:352 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.505320184999999, 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014313115, 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 352 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966779+0000", 2024-09-21T23:15:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.22e-07 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966783+0000", 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8099999999999997e-06 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966788+0000", 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.758e-06 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966791+0000", 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1489999999999998e-06 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980882+0000", 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014090965 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980969+0000", 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6222000000000002e-05 2024-09-21T23:15:30.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981091+0000", 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000122889 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:353 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.505302677, 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014443615, 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.512 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 353 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966802+0000", 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9070000000000004e-06 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966806+0000", 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1200000000000004e-06 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966813+0000", 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5239999999999997e-06 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981114+0000", 2024-09-21T23:15:30.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014301385 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981166+0000", 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2021999999999998e-05 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981239+0000", 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3385000000000003e-05 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.516 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:974 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 520192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.324759834000002, 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013957262999999999, 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 974 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:30.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147340+0000", 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2589999999999999e-06 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147344+0000", 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1899999999999997e-06 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147348+0000", 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161133+0000", 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013784597000000001 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161205+0000", 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2692000000000001e-05 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161296+0000", 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0550000000000005e-05 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2367 3.16 3:6d83b35b:::rbd_object_map.10f16a447f06:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.958748725, 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015679904000000001, 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2367 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513351+0000", 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513355+0000", 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5929999999999999e-06 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513372+0000", 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7062999999999999e-05 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513386+0000", 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3293e-05 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523249+0000", 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.009863251 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523437+0000", 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018793 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.524029+0000", 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059259600000000003 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.525598+0000", 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001569017 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528769+0000", 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003170754 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528893+0000", 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012397100000000001 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.529030+0000", 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013660899999999999 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2583 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.917486440000001, 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013835072, 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.526 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2583 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1e-07 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554613+0000", 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.212e-06 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554617+0000", 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554619+0000", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6680000000000001e-06 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568239+0000", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013620098000000001 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568331+0000", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1255999999999995e-05 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568447+0000", 2024-09-21T23:15:30.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011650399999999999 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2584 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.917475956000001, 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014048454, 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2584 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4999999999999999e-07 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554624+0000", 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.119e-06 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554627+0000", 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1200000000000002e-06 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554630+0000", 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7219999999999999e-06 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568490+0000", 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013860268 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568570+0000", 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9903999999999999e-05 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568671+0000", 2024-09-21T23:15:30.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101071 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2585 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.917465571000001, 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014283607, 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2585 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.77e-07 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554634+0000", 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.144e-06 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554638+0000", 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2440000000000002e-06 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554640+0000", 2024-09-21T23:15:30.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6869999999999999e-06 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568717+0000", 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014077118 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568802+0000", 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4079000000000006e-05 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568917+0000", 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011495800000000001 2024-09-21T23:15:30.537 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2586 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 21.917441546999999, 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014464238000000001, 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2586 2024-09-21T23:15:30.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.28e-07 2024-09-21T23:15:30.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554658+0000", 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.282e-06 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554662+0000", 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5470000000000001e-06 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554665+0000", 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568947+0000", 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01428159 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569026+0000", 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8950999999999997e-05 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569121+0000", 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5173000000000005e-05 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:388 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836097+0000", 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6360013660000003, 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015437466, 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 388 2024-09-21T23:15:30.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836097+0000", 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836097+0000", 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836105+0000", 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5769999999999996e-06 2024-09-21T23:15:30.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836114+0000", 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3109999999999995e-06 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836157+0000", 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2574000000000003e-05 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836192+0000", 2024-09-21T23:15:30.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5027999999999997e-05 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.844648+0000", 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084561900000000006 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.844791+0000", 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014363099999999999 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.845384+0000", 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000592133 2024-09-21T23:15:30.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851166+0000", 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0057823789999999998 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851355+0000", 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000188922 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851436+0000", 2024-09-21T23:15:30.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1289e-05 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851535+0000", 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8431999999999995e-05 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:389 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836223+0000", 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6358749819999998, 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015457797000000001, 2024-09-21T23:15:30.547 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 389 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836223+0000", 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836223+0000", 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836225+0000", 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8530000000000001e-06 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836234+0000", 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5739999999999996e-06 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836258+0000", 2024-09-21T23:15:30.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3884e-05 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836297+0000", 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9453999999999999e-05 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.846771+0000", 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010473779000000001 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.846915+0000", 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144034 2024-09-21T23:15:30.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.847985+0000", 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010695800000000001 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851333+0000", 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033487790000000001 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851553+0000", 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021979599999999999 2024-09-21T23:15:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851625+0000", 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1449e-05 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851681+0000", 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6614999999999999e-05 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.552 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:390 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836358+0000", 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6357404750000004, 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020077536999999999, 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 390 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836358+0000", 2024-09-21T23:15:30.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836358+0000", 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836376+0000", 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7594000000000001e-05 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836386+0000", 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0193e-05 2024-09-21T23:15:30.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836413+0000", 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7364000000000001e-05 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836429+0000", 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5716999999999999e-05 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.849139+0000", 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012709651000000001 2024-09-21T23:15:30.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.849374+0000", 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000235317 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.850308+0000", 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00093457200000000005 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856120+0000", 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0058112010000000002 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856241+0000", 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121669 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856318+0000", 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6699000000000005e-05 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856436+0000", 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011756000000000001 2024-09-21T23:15:30.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:391 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836491+0000", 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6356077979999997, 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020212988000000001, 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:30.558 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 391 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836491+0000", 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836491+0000", 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836492+0000", 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0899999999999999e-06 2024-09-21T23:15:30.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836509+0000", 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6807e-05 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836531+0000", 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2240000000000001e-05 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836546+0000", 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5540999999999999e-05 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851776+0000", 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015229397 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851926+0000", 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014989100000000001 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.852867+0000", 2024-09-21T23:15:30.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094158600000000001 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856221+0000", 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033537390000000001 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856466+0000", 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024535300000000002 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856628+0000", 2024-09-21T23:15:30.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016166499999999999 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856704+0000", 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5679000000000005e-05 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:392 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836600+0000", 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 7.6354981019999997, 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021550758, 2024-09-21T23:15:30.563 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 392 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836600+0000", 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:30.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836600+0000", 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836602+0000", 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-06 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836610+0000", 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1599999999999998e-06 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836630+0000", 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0540999999999999e-05 2024-09-21T23:15:30.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836646+0000", 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.575e-05 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.854151+0000", 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017505007 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.854371+0000", 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000220306 2024-09-21T23:15:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.855290+0000", 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00091819799999999995 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.857943+0000", 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026538030000000002 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.857966+0000", 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2251000000000001e-05 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.858047+0000", 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1852000000000002e-05 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.858151+0000", 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010367299999999999 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:30.568 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:30.569 INFO:teuthology.orchestra.run.smithi170.stdout:1446 write 0xbb21a00 thru 0xbb30f46 (0xf547 bytes) 2024-09-21T23:15:30.569 INFO:teuthology.orchestra.run.smithi170.stdout:1447 punch from 0x9608600 to 0x9617000, (0xea00 bytes) 2024-09-21T23:15:30.569 INFO:teuthology.orchestra.run.smithi170.stdout:1450 trunc from 0xbb31000 to 0x340bc00 2024-09-21T23:15:30.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:30.570 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:30.568+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:30.577 INFO:teuthology.orchestra.run.smithi170.stdout:1451 trunc from 0x340bc00 to 0xdea6400 2024-09-21T23:15:30.598 INFO:teuthology.orchestra.run.smithi170.stdout:1452 write 0x5789600 thru 0x578a42c (0xe2d bytes) 2024-09-21T23:15:30.601 INFO:teuthology.orchestra.run.smithi170.stdout:1453 write 0x90600 thru 0x979f3 (0x73f4 bytes) 2024-09-21T23:15:30.602 INFO:teuthology.orchestra.run.smithi170.stdout:1456 punch from 0x7650200 to 0x7651e00, (0x1c00 bytes) 2024-09-21T23:15:30.603 INFO:teuthology.orchestra.run.smithi170.stdout:1457 write 0x87da800 thru 0x87e768d (0xce8e bytes) 2024-09-21T23:15:30.603 INFO:teuthology.orchestra.run.smithi170.stdout:1458 write 0x3ea3c00 thru 0x3eae1fb (0xa5fc bytes) 2024-09-21T23:15:30.603 INFO:teuthology.orchestra.run.smithi170.stdout:1461 read 0x3de6c00 thru 0x3dedfef (0x73f0 bytes) 2024-09-21T23:15:30.604 INFO:teuthology.orchestra.run.smithi170.stdout:1462 write 0xa954600 thru 0xa959432 (0x4e33 bytes) 2024-09-21T23:15:30.604 INFO:teuthology.orchestra.run.smithi170.stdout:1463 write 0x84a7200 thru 0x84aab06 (0x3907 bytes) 2024-09-21T23:15:30.604 INFO:teuthology.orchestra.run.smithi170.stdout:1467 write 0x9377800 thru 0x938357f (0xbd80 bytes) 2024-09-21T23:15:30.619 INFO:teuthology.orchestra.run.smithi170.stdout:1470 trunc from 0xdea6400 to 0x4e9ec00 2024-09-21T23:15:30.670 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:30.663+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:30.671 INFO:teuthology.orchestra.run.smithi170.stdout:1473 write 0x115e000 thru 0x1163b90 (0x5b91 bytes) 2024-09-21T23:15:30.695 INFO:teuthology.orchestra.run.smithi170.stdout:1476 trunc from 0x4e9ec00 to 0xd984600 2024-09-21T23:15:30.717 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4993 3.12 3.dc925672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:30.718537+0000", 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00026266199999999999, 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032025800000000002, 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4993 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:30.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718537+0000", 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718537+0000", 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718539+0000", 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9470000000000002e-06 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718549+0000", 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8030000000000008e-06 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718574+0000", 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4964e-05 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718588+0000", 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4083000000000001e-05 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:30.720 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:1397 3.12 3.dc925672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:30.718562+0000", 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00023729799999999999, 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00049206300000000005, 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:30.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1397 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718562+0000", 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718562+0000", 2024-09-21T23:15:30.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718565+0000", 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0359999999999998e-06 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718578+0000", 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2627e-05 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718614+0000", 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5620000000000001e-05 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718634+0000", 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0015999999999999e-05 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.724 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:1396 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=19b out=9b,call rbd.get_features in=24b out=16b,call rbd.get_flags in=20b out=8b,call rbd.get_snapcontext in=18b out=12b,call lock.get_info in=30b out=115b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:30.717530+0000", 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.001269512, 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017474789999999999, 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1396 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:30.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.717530+0000", 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.717530+0000", 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.717531+0000", 2024-09-21T23:15:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0899999999999999e-06 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.717539+0000", 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3799999999999994e-06 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.717561+0000", 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-05 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.717577+0000", 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5749000000000001e-05 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718445+0000", 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086799900000000005 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:30.718672+0000", 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227381 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:30.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:30.729 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 3 2024-09-21T23:15:30.729 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:30.729 INFO:teuthology.orchestra.run.smithi170.stdout:1477 trunc from 0xd984600 to 0xbc5d000 2024-09-21T23:15:30.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:30.747 INFO:teuthology.orchestra.run.smithi170.stdout:1478 read 0xf20800 thru 0xf274ec (0x6ced bytes) 2024-09-21T23:15:30.748 INFO:teuthology.orchestra.run.smithi170.stdout:1480 write 0x326c000 thru 0x3273e38 (0x7e39 bytes) 2024-09-21T23:15:30.773 INFO:teuthology.orchestra.run.smithi170.stdout:1481 write 0xcb5a000 thru 0xcb61e00 (0x7e01 bytes) 2024-09-21T23:15:30.776 INFO:teuthology.orchestra.run.smithi170.stdout:1482 flatten 2024-09-21T23:15:30.784 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:30.784+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:30.798 INFO:teuthology.orchestra.run.smithi170.stdout:1483 read 0x8375e00 thru 0x83812e2 (0xb4e3 bytes) 2024-09-21T23:15:30.799 INFO:teuthology.orchestra.run.smithi170.stdout:1489 write 0xacbb600 thru 0xacc8506 (0xcf07 bytes) 2024-09-21T23:15:30.800 INFO:teuthology.orchestra.run.smithi170.stdout:1490 read 0x1de2c00 thru 0x1de42f7 (0x16f8 bytes) 2024-09-21T23:15:30.802 INFO:teuthology.orchestra.run.smithi170.stdout:1493 trunc from 0xcb62000 to 0xd7e2e00 2024-09-21T23:15:30.823 INFO:teuthology.orchestra.run.smithi170.stdout:1494 read 0x53cb400 thru 0x53da91d (0xf51e bytes) 2024-09-21T23:15:30.852 INFO:teuthology.orchestra.run.smithi170.stdout:1496 trunc from 0xd7e2e00 to 0x2b39400 2024-09-21T23:15:30.871 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:30.867+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:30.877 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:30.877 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:30.877 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:30.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:30.910 INFO:teuthology.orchestra.run.smithi170.stdout:1498 write 0x7d78600 thru 0x7d7d155 (0x4b56 bytes) 2024-09-21T23:15:30.940 INFO:teuthology.orchestra.run.smithi170.stdout:1499 write 0xa445e00 thru 0xa44c240 (0x6441 bytes) 2024-09-21T23:15:30.941 INFO:teuthology.orchestra.run.smithi170.stdout:1501 punch from 0x7741c00 to 0x7743c00, (0x2000 bytes) 2024-09-21T23:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout:1503 write 0x8533400 thru 0x853dd63 (0xa964 bytes) 2024-09-21T23:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout:1504 read 0x3e53800 thru 0x3e5f195 (0xb996 bytes) 2024-09-21T23:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout:1507 read 0x97d2600 thru 0x97d66e2 (0x40e3 bytes) 2024-09-21T23:15:30.942 INFO:teuthology.orchestra.run.smithi170.stdout:1508 punch from 0x19e5600 to 0x19e7c00, (0x2600 bytes) 2024-09-21T23:15:30.943 INFO:teuthology.orchestra.run.smithi170.stdout:1509 write 0x186200 thru 0x194401 (0xe202 bytes) 2024-09-21T23:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout:1512 write 0x2adea00 thru 0x2ae0b98 (0x2199 bytes) 2024-09-21T23:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout:1517 read 0x6f4b600 thru 0x6f5abcd (0xf5ce bytes) 2024-09-21T23:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout:1519 read 0x2347400 thru 0x234c6d5 (0x52d6 bytes) 2024-09-21T23:15:30.944 INFO:teuthology.orchestra.run.smithi170.stdout:1520 read 0x5bd2e00 thru 0x5bdd0bd (0xa2be bytes) 2024-09-21T23:15:30.945 INFO:teuthology.orchestra.run.smithi170.stdout:1522 write 0x424aa00 thru 0x4250265 (0x5866 bytes) 2024-09-21T23:15:30.945 INFO:teuthology.orchestra.run.smithi170.stdout:1524 punch from 0x5da2800 to 0x5daae00, (0x8600 bytes) 2024-09-21T23:15:30.945 INFO:teuthology.orchestra.run.smithi170.stdout:1525 read 0x143c00 thru 0x15266f (0xea70 bytes) 2024-09-21T23:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout:1526 read 0x1f6b200 thru 0x1f6d7a0 (0x25a1 bytes) 2024-09-21T23:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout:1528 write 0x3259400 thru 0x325caa8 (0x36a9 bytes) 2024-09-21T23:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout:1531 read 0x72f0200 thru 0x72fd1ab (0xcfac bytes) 2024-09-21T23:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout:1532 write 0x2635c00 thru 0x264069d (0xaa9e bytes) 2024-09-21T23:15:30.948 INFO:teuthology.orchestra.run.smithi170.stdout:1536 write 0x77ab200 thru 0x77b4239 (0x903a bytes) 2024-09-21T23:15:30.949 INFO:teuthology.orchestra.run.smithi170.stdout:1539 read 0x56da000 thru 0x56db297 (0x1298 bytes) 2024-09-21T23:15:30.949 INFO:teuthology.orchestra.run.smithi170.stdout:1540 read 0x3f1e600 thru 0x3f2497b (0x637c bytes) 2024-09-21T23:15:30.949 INFO:teuthology.orchestra.run.smithi170.stdout:1543 punch from 0x257dc00 to 0x258ae00, (0xd200 bytes) 2024-09-21T23:15:30.950 INFO:teuthology.orchestra.run.smithi170.stdout:1544 write 0x8310600 thru 0x831ce1e (0xc81f bytes) 2024-09-21T23:15:30.958 INFO:teuthology.orchestra.run.smithi170.stdout:1546 trunc from 0xa44c400 to 0xe5ade00 2024-09-21T23:15:30.980 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:30.980+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:31.007 INFO:teuthology.orchestra.run.smithi170.stdout:1548 trunc from 0xe5ade00 to 0x7a9400 2024-09-21T23:15:31.064 INFO:teuthology.orchestra.run.smithi170.stdout:1549 write 0x5259200 thru 0x525d72f (0x4530 bytes) 2024-09-21T23:15:31.068 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.724059163, 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:31.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:31.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:31.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:31.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.956416802, 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:31.075 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:31.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:31.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.57710672, 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.449889708000001, 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.535125977, 2024-09-21T23:15:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:15:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:15:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:15:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:15:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:15:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:15:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:15:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:15:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:15:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:15:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:15:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:15:31.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:15:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:15:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:15:31.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:15:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:15:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:15:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:15:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:15:31.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:15:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:15:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:15:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.102 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.080173211, 2024-09-21T23:15:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:15:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:15:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:31.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:15:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:15:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:15:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:15:31.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:15:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:15:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:15:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:15:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:15:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:15:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:15:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:31.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:15:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:15:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.230146828000001, 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.115 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:15:31.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:15:31.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:15:31.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:15:31.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.23010541, 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.120 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.048019483999999, 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:15:31.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:15:31.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:15:31.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:15:31.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:15:31.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.9060771410000008, 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.131 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:31.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:15:31.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:15:31.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18444 3.4 3:2145313d:::rbd_data.10f1e443a6fa.000000000000017b:head [set-alloc-hint object_size 524288 write_size 524288,write 278528~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6997505920000009, 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046378829999999998, 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18444 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369505+0000", 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369508+0000", 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8569999999999999e-06 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369539+0000", 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1108000000000001e-05 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369580+0000", 2024-09-21T23:15:31.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0679999999999997e-05 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369602+0000", 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1746000000000002e-05 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369735+0000", 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133349 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.369934+0000", 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000199507 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.370101+0000", 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166119 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.371736+0000", 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016352059999999999 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.373849+0000", 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002113326 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.374015+0000", 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016577200000000001 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.374143+0000", 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012821299999999999 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7502108459999999, 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:15:31.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5874805690000002, 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:15:31.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:15:31.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:15:31.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:15:31.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0680551859999996, 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:31.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:15:31.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:15:31.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0679267100000001, 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.159 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:15:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:15:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0669650060000002, 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0659537950000004, 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0648718480000001, 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:15:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:15:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:15:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.18829359, 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:15:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:15:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.1879540200000001, 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:15:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.187 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:31.189 INFO:teuthology.orchestra.run.smithi170.stdout:1550 write 0x5f0c800 thru 0x5f15b78 (0x9379 bytes) 2024-09-21T23:15:31.189 INFO:teuthology.orchestra.run.smithi170.stdout:1551 trunc from 0x5f15c00 to 0x455200 2024-09-21T23:15:31.189 INFO:teuthology.orchestra.run.smithi170.stdout:1552 clone 6 order 22 su 16384 sc 12 2024-09-21T23:15:31.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:31.190 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:31.188+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:31.291 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:31.287+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:31.367 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:31.367 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:31.367 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:31.367 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:31.381 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:31.392 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:31.387+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:31.492 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:31.487+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:31.502 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:31.502 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:31.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:31.540 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-21T23:15:31.540 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:15:31.540 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 4, 5] 2024-09-21T23:15:31.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-21T23:15:31.608 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:31.608+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:31.649 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.649 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17932 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.508857272, 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041236575999999997, 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17932 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138423+0000", 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138428+0000", 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7929999999999997e-06 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138438+0000", 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0672999999999999e-05 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138446+0000", 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4200000000000001e-06 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.171369+0000", 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032922699999999999 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.171556+0000", 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000187071 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172087+0000", 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053124200000000002 2024-09-21T23:15:31.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179510+0000", 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074233470000000003 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179565+0000", 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5047999999999999e-05 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179617+0000", 2024-09-21T23:15:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1579999999999997e-05 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179659+0000", 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1949000000000001e-05 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17933 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.50882389, 2024-09-21T23:15:31.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041549861, 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17933 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138461+0000", 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.33e-06 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138470+0000", 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6239999999999997e-06 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138478+0000", 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.173660+0000", 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035182577 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.173815+0000", 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154759 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.174562+0000", 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074730300000000005 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179805+0000", 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005243195 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179835+0000", 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9431e-05 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179918+0000", 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3591999999999995e-05 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180005+0000", 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7033999999999994e-05 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17934 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.50878646, 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045630958999999999, 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17934 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138494+0000", 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.23e-07 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138498+0000", 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.374e-06 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138508+0000", 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0135999999999999e-05 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138518+0000", 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5140000000000007e-06 2024-09-21T23:15:31.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176013+0000", 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.037495095999999999 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176109+0000", 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5759000000000003e-05 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176830+0000", 2024-09-21T23:15:31.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072154200000000004 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180029+0000", 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031987000000000001 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.183898+0000", 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038689639999999999 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184014+0000", 2024-09-21T23:15:31.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000116304 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184124+0000", 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010984699999999999 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17935 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.508751383, 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045851705999999999, 2024-09-21T23:15:31.666 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17935 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138529+0000", 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000002e-07 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138533+0000", 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8829999999999999e-06 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138558+0000", 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5123999999999999e-05 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138566+0000", 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1259999999999998e-06 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178007+0000", 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.039441213000000003 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178100+0000", 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2608000000000005e-05 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178652+0000", 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055207499999999996 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184230+0000", 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055785110000000004 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184245+0000", 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4958e-05 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184301+0000", 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5742000000000003e-05 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184380+0000", 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8832999999999996e-05 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17936 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.50870061, 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046903247000000002, 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17936 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.672 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138580+0000", 2024-09-21T23:15:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138584+0000", 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5940000000000004e-06 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138594+0000", 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0316000000000001e-05 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138602+0000", 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.17e-06 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180186+0000", 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041584401999999999 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180422+0000", 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023555099999999999 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.181094+0000", 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067216599999999997 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184518+0000", 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034243770000000001 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185304+0000", 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078624499999999998 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185406+0000", 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101691 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185482+0000", 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6081999999999994e-05 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17937 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.508663124, 2024-09-21T23:15:31.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047158114000000001, 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17937 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138617+0000", 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2399999999999997e-07 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138622+0000", 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5530000000000003e-06 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138632+0000", 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0096e-05 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138639+0000", 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1640000000000004e-06 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.182451+0000", 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.043812140999999999 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.182571+0000", 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012012 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.183035+0000", 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046411899999999998 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185503+0000", 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024672840000000001 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185610+0000", 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107126 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185720+0000", 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010995 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185775+0000", 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4836999999999997e-05 2024-09-21T23:15:31.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18434 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.295967357, 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040579337, 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18434 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351313+0000", 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351317+0000", 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9260000000000002e-06 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351326+0000", 2024-09-21T23:15:31.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0089999999999996e-06 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351333+0000", 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2169999999999997e-06 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.382673+0000", 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.031340149999999997 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.382938+0000", 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000264567 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.383955+0000", 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001017783 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391521+0000", 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075656059999999999 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391668+0000", 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014702399999999999 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391830+0000", 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000162447 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391891+0000", 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1033999999999999e-05 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18435 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.295937091000001, 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042792912000000002, 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18435 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351343+0000", 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.68e-07 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351347+0000", 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4569999999999998e-06 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351357+0000", 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0629999999999995e-06 2024-09-21T23:15:31.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351365+0000", 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4800000000000001e-06 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.386383+0000", 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035018214999999998 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.386544+0000", 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016093999999999999 2024-09-21T23:15:31.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.387525+0000", 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098102499999999995 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392029+0000", 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045039879999999996 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.393961+0000", 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001931552 2024-09-21T23:15:31.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394042+0000", 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1255999999999996e-05 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394135+0000", 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3368000000000005e-05 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18436 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.295905251000001, 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044976601999999997, 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18436 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.695 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351375+0000", 2024-09-21T23:15:31.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7100000000000002e-07 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351379+0000", 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351389+0000", 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351396+0000", 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.389268+0000", 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.037871888999999999 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.389448+0000", 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018001900000000001 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.390406+0000", 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095752800000000002 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396081+0000", 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0056750050000000003 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396116+0000", 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5160000000000002e-05 2024-09-21T23:15:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396254+0000", 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137954 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396351+0000", 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7076000000000004e-05 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18437 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.295872789000001, 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047245167999999997, 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18437 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7199999999999999e-07 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351412+0000", 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3200000000000001e-06 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351421+0000", 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0280000000000006e-06 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351428+0000", 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2490000000000004e-06 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392130+0000", 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040701945000000003 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392258+0000", 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012822500000000001 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392878+0000", 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062008699999999996 2024-09-21T23:15:31.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396380+0000", 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035021879999999998 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398426+0000", 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002046141 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398563+0000", 2024-09-21T23:15:31.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013656200000000001 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398652+0000", 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8850999999999993e-05 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18438 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.2958382, 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047837194, 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18438 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351442+0000", 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0200000000000002e-07 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351447+0000", 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5979999999999999e-06 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351457+0000", 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9850000000000001e-06 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351464+0000", 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394225+0000", 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042761087000000003 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394351+0000", 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012576199999999999 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394839+0000", 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048866800000000005 2024-09-21T23:15:31.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398696+0000", 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038569419999999999 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399092+0000", 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039590699999999998 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399184+0000", 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2386000000000006e-05 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399279+0000", 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4024999999999995e-05 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.712 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18754 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.103292806000001, 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040550092000000003, 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18754 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543990+0000", 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543997+0000", 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7880000000000001e-06 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544002+0000", 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:15:31.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578134+0000", 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.034132205999999998 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578328+0000", 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019426299999999999 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578872+0000", 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054318300000000003 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582350+0000", 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034784690000000001 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584435+0000", 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020848820000000001 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584490+0000", 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4765999999999997e-05 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584537+0000", 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7111999999999999e-05 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.718 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18756 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.103270357, 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042644648, 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18756 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544010+0000", 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544013+0000", 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4910000000000001e-06 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544020+0000", 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8639999999999998e-06 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544025+0000", 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4469999999999997e-06 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580135+0000", 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036109943999999998 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580291+0000", 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015522600000000001 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580766+0000", 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047577900000000001 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586496+0000", 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057296990000000004 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586516+0000", 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0326000000000002e-05 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586576+0000", 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.965e-05 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586654+0000", 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7775000000000007e-05 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18758 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:31.724 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.103139132000001, 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042728264000000002, 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18758 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544141+0000", 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2099999999999995e-07 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544146+0000", 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9559999999999996e-06 2024-09-21T23:15:31.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544157+0000", 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0465999999999999e-05 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544163+0000", 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4339999999999997e-06 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582457+0000", 2024-09-21T23:15:31.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.038293466999999998 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582614+0000", 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015732699999999999 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.583297+0000", 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000682724 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586742+0000", 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003444964 2024-09-21T23:15:31.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586755+0000", 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3689999999999999e-05 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586802+0000", 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6841000000000001e-05 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586869+0000", 2024-09-21T23:15:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6574000000000003e-05 2024-09-21T23:15:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18759 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.103007156, 2024-09-21T23:15:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047298266999999998, 2024-09-21T23:15:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.731 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18759 2024-09-21T23:15:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544273+0000", 2024-09-21T23:15:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4600000000000004e-07 2024-09-21T23:15:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544278+0000", 2024-09-21T23:15:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0579999999999997e-06 2024-09-21T23:15:31.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544288+0000", 2024-09-21T23:15:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0104e-05 2024-09-21T23:15:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544295+0000", 2024-09-21T23:15:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4869999999999999e-06 2024-09-21T23:15:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584612+0000", 2024-09-21T23:15:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040317510000000001 2024-09-21T23:15:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584745+0000", 2024-09-21T23:15:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133051 2024-09-21T23:15:31.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.585225+0000", 2024-09-21T23:15:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000480054 2024-09-21T23:15:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591274+0000", 2024-09-21T23:15:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060487429999999997 2024-09-21T23:15:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591414+0000", 2024-09-21T23:15:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014027700000000001 2024-09-21T23:15:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591498+0000", 2024-09-21T23:15:31.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3231000000000003e-05 2024-09-21T23:15:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591571+0000", 2024-09-21T23:15:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3005999999999999e-05 2024-09-21T23:15:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18760 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:31.743 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.102975435999999, 2024-09-21T23:15:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047443218000000002, 2024-09-21T23:15:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18760 2024-09-21T23:15:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:31.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544305+0000", 2024-09-21T23:15:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:15:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544308+0000", 2024-09-21T23:15:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3680000000000002e-06 2024-09-21T23:15:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544315+0000", 2024-09-21T23:15:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000003e-06 2024-09-21T23:15:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544321+0000", 2024-09-21T23:15:31.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.507e-06 2024-09-21T23:15:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586945+0000", 2024-09-21T23:15:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042624414999999999 2024-09-21T23:15:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.587111+0000", 2024-09-21T23:15:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016594099999999999 2024-09-21T23:15:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.587753+0000", 2024-09-21T23:15:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064202200000000001 2024-09-21T23:15:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591388+0000", 2024-09-21T23:15:31.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036342829999999999 2024-09-21T23:15:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591591+0000", 2024-09-21T23:15:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020310700000000001 2024-09-21T23:15:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591679+0000", 2024-09-21T23:15:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8796999999999997e-05 2024-09-21T23:15:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591747+0000", 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7879999999999994e-05 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18761 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.102948702999999, 2024-09-21T23:15:31.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053139912999999997, 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18761 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544335+0000", 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:31.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544342+0000", 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9879999999999998e-06 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544347+0000", 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.589606+0000", 2024-09-21T23:15:31.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045258537000000001 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.589780+0000", 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017489299999999999 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.590363+0000", 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058220899999999998 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597288+0000", 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069252100000000002 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597312+0000", 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4057e-05 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597396+0000", 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4449000000000004e-05 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597471+0000", 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.436e-05 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18762 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.10292557, 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.05334962, 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18762 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3599999999999999e-07 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544361+0000", 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0269999999999998e-06 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544372+0000", 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0359e-05 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544378+0000", 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8139999999999997e-06 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591881+0000", 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047503385000000002 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.592076+0000", 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019506100000000001 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.592696+0000", 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062034199999999996 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597489+0000", 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047924430000000004 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597556+0000", 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7407e-05 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597653+0000", 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7225000000000005e-05 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597704+0000", 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0120999999999997e-05 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18763 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.102893974000001, 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053606662999999999, 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18763 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544389+0000", 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2119999999999999e-06 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544397+0000", 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2910000000000002e-06 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544402+0000", 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:15:31.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.593900+0000", 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049497541999999999 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.594001+0000", 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101384 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.594385+0000", 2024-09-21T23:15:31.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038437500000000001 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597728+0000", 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003342303 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597831+0000", 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102977 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597927+0000", 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6080000000000002e-05 2024-09-21T23:15:31.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597992+0000", 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5408999999999997e-05 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18764 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.102870469999999, 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053769346000000003, 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:31.772 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18764 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:31.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3300000000000003e-07 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544413+0000", 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.236e-06 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544420+0000", 2024-09-21T23:15:31.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8859999999999998e-06 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544425+0000", 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.48e-06 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.595366+0000", 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050940791999999999 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:31.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.595531+0000", 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016486100000000001 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.596215+0000", 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068378600000000003 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598024+0000", 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001809361 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598080+0000", 2024-09-21T23:15:31.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6538e-05 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598135+0000", 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4703000000000001e-05 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598178+0000", 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3269999999999997e-05 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:31.777 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:31.779 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:31.779 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:31.775+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:31.867 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-21T23:15:31.864+0000 7faab395a640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2024-09-21T23:15:31.879 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:31.875+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:31.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:31.909 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:31.909 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:31.909 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:31.909 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:31.923 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.1. 2024-09-21T23:15:32.011 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:32.007+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:32.016 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:32.027 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:15:32.033 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:32.034 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:32.050 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:32.051 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:32.079 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:32.079 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:32.080 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:32.075+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:32.089 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:32.089 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:32.098 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:32.098 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:32.105 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:32.119 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:32.124 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:32.191 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:32.187+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:32.265 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:32.265 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:32.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:32.281 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:32.280+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:32.383 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:32.379+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:32.442 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:32.442 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:32.456 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:32.499 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:32.495+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:32.585 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:32.584+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:32.624 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:32.624 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:32.638 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:32.685 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:32.679+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:32.759 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:32.759 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:32.773 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:32.787 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:32.784+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:32.888 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:32.883+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:15:32.945 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:32.945 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:32.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:33.007 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:33.003+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:33.089 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:33.083+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:33.124 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:33.124 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:33.138 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:33.190 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:33.188+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:33.291 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:33.287+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:33.305 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:33.305 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:33.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:33.392 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:33.388+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:33.453 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:33.453 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:33.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:33.512 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:33.512+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:33.593 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:33.587+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:33.633 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:33.633 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:33.648 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:33.693 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:33.687+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:33.773 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:33.773 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:33.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:33.794 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:33.787+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:33.895 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:33.892+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:33.914 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:33.914 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:33.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:33.995 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:33.991+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:34.096 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:34.091+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:34.113 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:34.113 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:34.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:34.128 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:34.197 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:34.191+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:34.267 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:34.267 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:34.298 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:34.291+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:34.301 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:34.301 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:34.327 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17932 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.187237414, 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041236575999999997, 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17932 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138422+0000", 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138423+0000", 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138428+0000", 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7929999999999997e-06 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138438+0000", 2024-09-21T23:15:34.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0672999999999999e-05 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138446+0000", 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4200000000000001e-06 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.171369+0000", 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.032922699999999999 2024-09-21T23:15:34.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.171556+0000", 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000187071 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.172087+0000", 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053124200000000002 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179510+0000", 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074233470000000003 2024-09-21T23:15:34.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179565+0000", 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5047999999999999e-05 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179617+0000", 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1579999999999997e-05 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179659+0000", 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1949000000000001e-05 2024-09-21T23:15:34.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17933 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.187204032, 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041549861, 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17933 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138456+0000", 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138461+0000", 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.33e-06 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138470+0000", 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6239999999999997e-06 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138478+0000", 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.173660+0000", 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035182577 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.173815+0000", 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154759 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.174562+0000", 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074730300000000005 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179805+0000", 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005243195 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179835+0000", 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9431e-05 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.179918+0000", 2024-09-21T23:15:34.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3591999999999995e-05 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180005+0000", 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7033999999999994e-05 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17934 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.187166602, 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045630958999999999, 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17934 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138493+0000", 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138494+0000", 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.23e-07 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138498+0000", 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.374e-06 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138508+0000", 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0135999999999999e-05 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138518+0000", 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5140000000000007e-06 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176013+0000", 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.037495095999999999 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176109+0000", 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5759000000000003e-05 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.176830+0000", 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072154200000000004 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180029+0000", 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031987000000000001 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.183898+0000", 2024-09-21T23:15:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038689639999999999 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184014+0000", 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000116304 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184124+0000", 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010984699999999999 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17935 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.187131525, 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045851705999999999, 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17935 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.345 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138528+0000", 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138529+0000", 2024-09-21T23:15:34.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000002e-07 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138533+0000", 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8829999999999999e-06 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138558+0000", 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5123999999999999e-05 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138566+0000", 2024-09-21T23:15:34.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1259999999999998e-06 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178007+0000", 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.039441213000000003 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178100+0000", 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2608000000000005e-05 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.178652+0000", 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055207499999999996 2024-09-21T23:15:34.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184230+0000", 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055785110000000004 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184245+0000", 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4958e-05 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184301+0000", 2024-09-21T23:15:34.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5742000000000003e-05 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184380+0000", 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8832999999999996e-05 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17936 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.187080752, 2024-09-21T23:15:34.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.046903247000000002, 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17936 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138579+0000", 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138580+0000", 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5300000000000004e-07 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138584+0000", 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5940000000000004e-06 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138594+0000", 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0316000000000001e-05 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138602+0000", 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.17e-06 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180186+0000", 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.041584401999999999 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.180422+0000", 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023555099999999999 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.181094+0000", 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067216599999999997 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.184518+0000", 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034243770000000001 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185304+0000", 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078624499999999998 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185406+0000", 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101691 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185482+0000", 2024-09-21T23:15:34.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6081999999999994e-05 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17937 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.187043266, 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047158114000000001, 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17937 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138616+0000", 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138617+0000", 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2399999999999997e-07 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138622+0000", 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5530000000000003e-06 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138632+0000", 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0096e-05 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.138639+0000", 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1640000000000004e-06 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.182451+0000", 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.043812140999999999 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.182571+0000", 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012012 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.183035+0000", 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046411899999999998 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185503+0000", 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024672840000000001 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185610+0000", 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107126 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185720+0000", 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010995 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.185775+0000", 2024-09-21T23:15:34.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4836999999999997e-05 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18434 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.974347499, 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040579337, 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18434 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351312+0000", 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351313+0000", 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351317+0000", 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9260000000000002e-06 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351326+0000", 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0089999999999996e-06 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351333+0000", 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2169999999999997e-06 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.382673+0000", 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.031340149999999997 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.382938+0000", 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000264567 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.383955+0000", 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001017783 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391521+0000", 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075656059999999999 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391668+0000", 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014702399999999999 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391830+0000", 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000162447 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.391891+0000", 2024-09-21T23:15:34.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1033999999999999e-05 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18435 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.974317233000001, 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042792912000000002, 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.368 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18435 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351342+0000", 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351343+0000", 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.68e-07 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351347+0000", 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4569999999999998e-06 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351357+0000", 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0629999999999995e-06 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351365+0000", 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4800000000000001e-06 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.386383+0000", 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.035018214999999998 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.386544+0000", 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016093999999999999 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.387525+0000", 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098102499999999995 2024-09-21T23:15:34.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392029+0000", 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045039879999999996 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.393961+0000", 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001931552 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394042+0000", 2024-09-21T23:15:34.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1255999999999996e-05 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394135+0000", 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3368000000000005e-05 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18436 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:34.374 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.974285393000001, 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.044976601999999997, 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18436 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:34.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351374+0000", 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351375+0000", 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7100000000000002e-07 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351379+0000", 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351389+0000", 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0397000000000001e-05 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351396+0000", 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.389268+0000", 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.037871888999999999 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.389448+0000", 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018001900000000001 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.390406+0000", 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095752800000000002 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396081+0000", 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0056750050000000003 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396116+0000", 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5160000000000002e-05 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396254+0000", 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137954 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396351+0000", 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7076000000000004e-05 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18437 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.974252931000001, 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047245167999999997, 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18437 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351407+0000", 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7199999999999999e-07 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351412+0000", 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3200000000000001e-06 2024-09-21T23:15:34.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351421+0000", 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0280000000000006e-06 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351428+0000", 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2490000000000004e-06 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392130+0000", 2024-09-21T23:15:34.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040701945000000003 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392258+0000", 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012822500000000001 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.392878+0000", 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062008699999999996 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.396380+0000", 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035021879999999998 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398426+0000", 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002046141 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398563+0000", 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013656200000000001 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398652+0000", 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8850999999999993e-05 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18438 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.974218342, 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047837194, 2024-09-21T23:15:34.386 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18438 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351441+0000", 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351442+0000", 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0200000000000002e-07 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351447+0000", 2024-09-21T23:15:34.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5979999999999999e-06 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351457+0000", 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9850000000000001e-06 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.351464+0000", 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394225+0000", 2024-09-21T23:15:34.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042761087000000003 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394351+0000", 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012576199999999999 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.394839+0000", 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048866800000000005 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.398696+0000", 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038569419999999999 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399092+0000", 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039590699999999998 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399184+0000", 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2386000000000006e-05 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.399279+0000", 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4024999999999995e-05 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18754 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781672948000001, 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040550092000000003, 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18754 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543987+0000", 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543990+0000", 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0599999999999999e-06 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.543997+0000", 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7880000000000001e-06 2024-09-21T23:15:34.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544002+0000", 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9309999999999999e-06 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578134+0000", 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.034132205999999998 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578328+0000", 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019426299999999999 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.578872+0000", 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054318300000000003 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582350+0000", 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034784690000000001 2024-09-21T23:15:34.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584435+0000", 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020848820000000001 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584490+0000", 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4765999999999997e-05 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584537+0000", 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7111999999999999e-05 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18756 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781650498999999, 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042644648, 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18756 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544009+0000", 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544010+0000", 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544013+0000", 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4910000000000001e-06 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544020+0000", 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8639999999999998e-06 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544025+0000", 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4469999999999997e-06 2024-09-21T23:15:34.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580135+0000", 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.036109943999999998 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580291+0000", 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015522600000000001 2024-09-21T23:15:34.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.580766+0000", 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047577900000000001 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586496+0000", 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057296990000000004 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586516+0000", 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0326000000000002e-05 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586576+0000", 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.965e-05 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586654+0000", 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7775000000000007e-05 2024-09-21T23:15:34.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18758 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781519274000001, 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042728264000000002, 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18758 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544140+0000", 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544141+0000", 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2099999999999995e-07 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544146+0000", 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9559999999999996e-06 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544157+0000", 2024-09-21T23:15:34.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0465999999999999e-05 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544163+0000", 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4339999999999997e-06 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582457+0000", 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.038293466999999998 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.582614+0000", 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015732699999999999 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.583297+0000", 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000682724 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586742+0000", 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003444964 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586755+0000", 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3689999999999999e-05 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586802+0000", 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6841000000000001e-05 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586869+0000", 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6574000000000003e-05 2024-09-21T23:15:34.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18759 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781387298, 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047298266999999998, 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18759 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544272+0000", 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544273+0000", 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4600000000000004e-07 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544278+0000", 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0579999999999997e-06 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544288+0000", 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0104e-05 2024-09-21T23:15:34.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544295+0000", 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4869999999999999e-06 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584612+0000", 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.040317510000000001 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.584745+0000", 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133051 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.585225+0000", 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000480054 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591274+0000", 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060487429999999997 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591414+0000", 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014027700000000001 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591498+0000", 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3231000000000003e-05 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591571+0000", 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3005999999999999e-05 2024-09-21T23:15:34.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18760 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781355577999999, 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047443218000000002, 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18760 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544304+0000", 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544305+0000", 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544308+0000", 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3680000000000002e-06 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544315+0000", 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000003e-06 2024-09-21T23:15:34.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544321+0000", 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.507e-06 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.586945+0000", 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.042624414999999999 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.587111+0000", 2024-09-21T23:15:34.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016594099999999999 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.587753+0000", 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064202200000000001 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591388+0000", 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036342829999999999 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591591+0000", 2024-09-21T23:15:34.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020310700000000001 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591679+0000", 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8796999999999997e-05 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591747+0000", 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7879999999999994e-05 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18761 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781328845000001, 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053139912999999997, 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18761 2024-09-21T23:15:34.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544331+0000", 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544335+0000", 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544342+0000", 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9879999999999998e-06 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544347+0000", 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3650000000000003e-06 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.589606+0000", 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.045258537000000001 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.589780+0000", 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017489299999999999 2024-09-21T23:15:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.590363+0000", 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058220899999999998 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597288+0000", 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069252100000000002 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597312+0000", 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4057e-05 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597396+0000", 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4449000000000004e-05 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597471+0000", 2024-09-21T23:15:34.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.436e-05 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18762 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781305712, 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.05334962, 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18762 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:34.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544354+0000", 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3599999999999999e-07 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544361+0000", 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0269999999999998e-06 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544372+0000", 2024-09-21T23:15:34.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0359e-05 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544378+0000", 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8139999999999997e-06 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.591881+0000", 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047503385000000002 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.592076+0000", 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019506100000000001 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.592696+0000", 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062034199999999996 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597489+0000", 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047924430000000004 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597556+0000", 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7407e-05 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597653+0000", 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7225000000000005e-05 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597704+0000", 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0120999999999997e-05 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18763 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781274116000001, 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053606662999999999, 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18763 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544386+0000", 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8999999999999997e-07 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544389+0000", 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2119999999999999e-06 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544397+0000", 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2910000000000002e-06 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544402+0000", 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5999999999999997e-06 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.593900+0000", 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.049497541999999999 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.594001+0000", 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101384 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.594385+0000", 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038437500000000001 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597728+0000", 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003342303 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597831+0000", 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102977 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597927+0000", 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6080000000000002e-05 2024-09-21T23:15:34.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.597992+0000", 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5408999999999997e-05 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:18764 3.7 3:e1bff944:::rbd_object_map.10f1e443a6fa:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.781250612000001, 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.053769346000000003, 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:34.444 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18764 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:34.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544409+0000", 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3300000000000003e-07 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544413+0000", 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.236e-06 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544420+0000", 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8859999999999998e-06 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.544425+0000", 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.48e-06 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.595366+0000", 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.050940791999999999 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.595531+0000", 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016486100000000001 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.596215+0000", 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068378600000000003 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598024+0000", 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001809361 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598080+0000", 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6538e-05 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598135+0000", 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4703000000000001e-05 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.598178+0000", 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3269999999999997e-05 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:34.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:34.450 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:34.451 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:34.451 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:34.452 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:34.452 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:34.452 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:34.453 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:34.453 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:34.453 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:34.454 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:34.454 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:34.447+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:34.552 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:34.552+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:34.565 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-21T23:15:34.584 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:34.584 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:34.584 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:34.585 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:34.598 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:34.653 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:34.647+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:34.718 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:34.718 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:34.718 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:34.718 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:34.718 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:34.733 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:34.775 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:34.771+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:34.786 INFO:teuthology.orchestra.run.smithi170.stdout:1553 write 0xecc8600 thru 0xecd0f57 (0x8958 bytes) 2024-09-21T23:15:34.788 INFO:teuthology.orchestra.run.smithi170.stdout:1554 read 0x9914000 thru 0x9915cd5 (0x1cd6 bytes) 2024-09-21T23:15:34.788 INFO:teuthology.orchestra.run.smithi170.stdout:1555 write 0x3aa3e00 thru 0x3aaeb73 (0xad74 bytes) 2024-09-21T23:15:34.789 INFO:teuthology.orchestra.run.smithi170.stdout:1556 read 0xc6a1a00 thru 0xc6a3628 (0x1c29 bytes) 2024-09-21T23:15:34.789 INFO:teuthology.orchestra.run.smithi170.stdout:1558 read 0x3a9ae00 thru 0x3aa6038 (0xb239 bytes) 2024-09-21T23:15:34.789 INFO:teuthology.orchestra.run.smithi170.stdout:1559 write 0x38c600 thru 0x3950b3 (0x8ab4 bytes) 2024-09-21T23:15:34.793 INFO:teuthology.orchestra.run.smithi170.stdout:1560 read 0x2eafe00 thru 0x2eb7afa (0x7cfb bytes) 2024-09-21T23:15:34.793 INFO:teuthology.orchestra.run.smithi170.stdout:1564 read 0xbb7ee00 thru 0xbb8ae81 (0xc082 bytes) 2024-09-21T23:15:34.793 INFO:teuthology.orchestra.run.smithi170.stdout:1566 read 0xcb85e00 thru 0xcb86acd (0xcce bytes) 2024-09-21T23:15:34.793 INFO:teuthology.orchestra.run.smithi170.stdout:1568 punch from 0x96d3800 to 0x96da400, (0x6c00 bytes) 2024-09-21T23:15:34.796 INFO:teuthology.orchestra.run.smithi170.stdout:1570 read 0xd12b800 thru 0xd13a899 (0xf09a bytes) 2024-09-21T23:15:34.796 INFO:teuthology.orchestra.run.smithi170.stdout:1572 read 0xe321c00 thru 0xe3285fb (0x69fc bytes) 2024-09-21T23:15:34.796 INFO:teuthology.orchestra.run.smithi170.stdout:1575 punch from 0x28b2e00 to 0x28b9c00, (0x6e00 bytes) 2024-09-21T23:15:34.815 INFO:teuthology.orchestra.run.smithi170.stdout:1576 trunc from 0xecd1000 to 0x3cb2400 2024-09-21T23:15:34.855 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:34.851+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:34.881 INFO:teuthology.orchestra.run.smithi170.stdout:1578 read 0x2a0d200 thru 0x2a0f64f (0x2450 bytes) 2024-09-21T23:15:34.891 INFO:teuthology.orchestra.run.smithi170.stdout:1579 trunc from 0x3cb2400 to 0x742a00 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.696639015999999, 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:34.919 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.184233732999999, 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:34.925 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.930 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.184168829999997, 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:34.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:34.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.936 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.184022390999999, 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:34.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:34.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.183953275999997, 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:34.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.948 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.183340950999998, 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:34.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:34.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:34.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.183136546, 2024-09-21T23:15:34.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:34.959 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.183093759999998, 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.960 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:34.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:34.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.965 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.792135471999998, 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:34.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:34.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.792048962000003, 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:34.971 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:34.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:34.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791972072999997, 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.982 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791767233000002, 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:34.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:34.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791678951000002, 2024-09-21T23:15:34.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:34.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:34.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:34.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:34.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791548122999998, 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:34.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:34.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:34.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:34.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:34.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:34.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791502469999998, 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:35.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791464042999998, 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:35.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:35.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:35.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791334581000001, 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791312939000001, 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.017 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:35.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:35.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:35.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:35.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791292894999998, 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:35.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.791208902999998, 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:35.028 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:35.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.034 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:35.035 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:15:35.036 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:35.036 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:35.031+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:35.136 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:35.131+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:35.208 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:35.208 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3.a1812e1a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.25288240200000001, 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.252926659, 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for peered", 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:15:35.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:15:35.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:15:35.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:15:35.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3.a1812e1a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.25261541799999998, 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.252801793, 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.213 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for peered", 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:15:35.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.217 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:15:35.218 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 2 2024-09-21T23:15:35.218 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:35.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:35.267 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:35.263+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:35.337 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:35.336+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:35.406 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:35.455 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:35.451+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2008 3.3 3:c4d74490:::rbd_data.10f16a447f06.00000000000000b1:head [set-alloc-hint object_size 524288 write_size 524288,write 354304~14336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.143130473999999, 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088517769999999999, 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.533 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2008 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383974+0000", 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.383977+0000", 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.959e-06 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384014+0000", 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6659999999999998e-05 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384039+0000", 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5745000000000002e-05 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384057+0000", 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7575999999999999e-05 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384162+0000", 2024-09-21T23:15:35.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010501 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.384169+0000", 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8170000000000001e-06 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390619+0000", 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0064506559999999999 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390698+0000", 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8777999999999997e-05 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.390782+0000", 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3552000000000002e-05 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.391690+0000", 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000908609 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392719+0000", 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001029051 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392782+0000", 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2644999999999994e-05 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.392826+0000", 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3718999999999999e-05 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.540 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:255 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.2154064130000002, 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088296720000000002, 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 255 2024-09-21T23:15:35.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:35.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311698+0000", 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311701+0000", 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9380000000000001e-06 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311712+0000", 2024-09-21T23:15:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0421999999999999e-05 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311719+0000", 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4730000000000003e-06 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315379+0000", 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00365936 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315427+0000", 2024-09-21T23:15:35.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8838000000000002e-05 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320372+0000", 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049444349999999996 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320474+0000", 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101902 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320528+0000", 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3940999999999997e-05 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:256 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.2153720069999991, 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0088795980000000007, 2024-09-21T23:15:35.546 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 256 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311732+0000", 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311733+0000", 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311738+0000", 2024-09-21T23:15:35.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.989e-06 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311748+0000", 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5370000000000003e-06 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.311754+0000", 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9730000000000002e-06 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.315953+0000", 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0041992679999999999 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.316028+0000", 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5337000000000001e-05 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320534+0000", 2024-09-21T23:15:35.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045061620000000002 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320567+0000", 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2251e-05 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.320612+0000", 2024-09-21T23:15:35.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5425e-05 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:349 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.1498089740000008, 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0093055059999999998, 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.552 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 349 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377295+0000", 2024-09-21T23:15:35.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377296+0000", 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3499999999999996e-07 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377301+0000", 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4270000000000001e-06 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377309+0000", 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3899999999999993e-06 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.377315+0000", 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4200000000000004e-06 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.380850+0000", 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035341690000000002 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.380984+0000", 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134857 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386492+0000", 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055073939999999997 2024-09-21T23:15:35.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386540+0000", 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8424999999999998e-05 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386601+0000", 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0788999999999997e-05 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:354 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.1488058520000006, 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096360100000000004, 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 354 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378299+0000", 2024-09-21T23:15:35.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8999999999999996e-07 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378304+0000", 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4479999999999996e-06 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378314+0000", 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.028e-05 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378321+0000", 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6340000000000003e-06 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.384551+0000", 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0062308509999999999 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.384845+0000", 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029336299999999999 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387768+0000", 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0029234249999999999 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387860+0000", 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2260000000000001e-05 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.387935+0000", 2024-09-21T23:15:35.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4159000000000006e-05 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:355 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.1486266020000002, 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010205018999999999, 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 355 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378478+0000", 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378479+0000", 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3600000000000003e-07 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378484+0000", 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3680000000000001e-06 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378497+0000", 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3086e-05 2024-09-21T23:15:35.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.378504+0000", 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2509999999999997e-06 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385893+0000", 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073889749999999999 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.385942+0000", 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9002999999999998e-05 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388509+0000", 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002567036 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388601+0000", 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000001e-05 2024-09-21T23:15:35.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388683+0000", 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2293999999999995e-05 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4510.0:356 3.1b 3:df8eb827:::rbd_object_map.10f192b3a1cb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.1479105549999993, 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095977030000000008, 2024-09-21T23:15:35.568 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4510", 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 356 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379194+0000", 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379195+0000", 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7400000000000002e-07 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379200+0000", 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3229999999999997e-06 2024-09-21T23:15:35.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379212+0000", 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1677e-05 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.379219+0000", 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4959999999999999e-06 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386720+0000", 2024-09-21T23:15:35.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.007500657 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.386785+0000", 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4667999999999998e-05 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388690+0000", 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001905452 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388726+0000", 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5565999999999998e-05 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:26.388792+0000", 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5989999999999997e-05 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1326 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.618398+0000", 2024-09-21T23:15:35.573 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9087064199999997, 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090124190000000007, 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1326 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618398+0000", 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618398+0000", 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618399+0000", 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9600000000000008e-07 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618405+0000", 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.981e-06 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618418+0000", 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2999999999999999e-05 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618427+0000", 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1269999999999992e-06 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.620060+0000", 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00163283 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.620236+0000", 2024-09-21T23:15:35.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017562 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.621024+0000", 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078876599999999999 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.626944+0000", 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059196789999999997 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627093+0000", 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014909499999999999 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627258+0000", 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016459299999999999 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627410+0000", 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015273199999999999 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1327 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.618510+0000", 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.908594345, 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092030859999999992, 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.580 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1327 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618510+0000", 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618510+0000", 2024-09-21T23:15:35.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618511+0000", 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5000000000000001e-07 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618518+0000", 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5869999999999998e-06 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618531+0000", 2024-09-21T23:15:35.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3057000000000001e-05 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618540+0000", 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0969999999999995e-06 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.622077+0000", 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035367749999999998 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.622328+0000", 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025092099999999998 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.623129+0000", 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00080162600000000005 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627555+0000", 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0044257080000000004 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627577+0000", 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2033000000000001e-05 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627654+0000", 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6728000000000001e-05 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627713+0000", 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9604000000000001e-05 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1328 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.618554+0000", 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9085502979999998, 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010989797000000001, 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1328 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.586 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618554+0000", 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618554+0000", 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618555+0000", 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0999999999999997e-07 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618560+0000", 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.42e-06 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618570+0000", 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0499999999999999e-05 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.618578+0000", 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6909999999999997e-06 2024-09-21T23:15:35.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.623966+0000", 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005388599 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.624225+0000", 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025900800000000002 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.625326+0000", 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011006919999999999 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.627876+0000", 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0025500319999999998 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.629349+0000", 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014731010000000001 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.629473+0000", 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012398300000000001 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.629544+0000", 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0860999999999998e-05 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1698 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.940551+0000", 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.5865536379999998, 2024-09-21T23:15:35.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011193362, 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1698 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.940551+0000", 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.940551+0000", 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.940552+0000", 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5999999999999999e-06 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.940564+0000", 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1137e-05 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.940595+0000", 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1130999999999997e-05 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.940610+0000", 2024-09-21T23:15:35.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4926000000000001e-05 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.940760+0000", 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015013900000000001 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.941023+0000", 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026286199999999999 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.942047+0000", 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010244169999999999 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.947373+0000", 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005326351 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.951599+0000", 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0042257170000000004 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.951678+0000", 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8807000000000006e-05 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.951744+0000", 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6274999999999998e-05 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1702 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.941513+0000", 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.5855917990000004, 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010479280000000001, 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1702 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.598 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.941513+0000", 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.941513+0000", 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.941514+0000", 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1200000000000001e-06 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.941522+0000", 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9240000000000007e-06 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.941541+0000", 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9357000000000001e-05 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.941552+0000", 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0543999999999999e-05 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943271+0000", 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001719581 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943509+0000", 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023808400000000001 2024-09-21T23:15:35.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.944423+0000", 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00091349200000000001 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.951761+0000", 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073380850000000003 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.951836+0000", 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5438000000000005e-05 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.951923+0000", 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6336999999999997e-05 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.951992+0000", 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9318000000000003e-05 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.603 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1709 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.943793+0000", 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.5833111439999996, 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090237089999999995, 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.604 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1709 2024-09-21T23:15:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943793+0000", 2024-09-21T23:15:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943793+0000", 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943795+0000", 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.55e-06 2024-09-21T23:15:35.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943804+0000", 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4900000000000006e-06 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943827+0000", 2024-09-21T23:15:35.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2917000000000001e-05 2024-09-21T23:15:35.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943844+0000", 2024-09-21T23:15:35.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6453999999999999e-05 2024-09-21T23:15:35.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.947503+0000", 2024-09-21T23:15:35.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036594330000000001 2024-09-21T23:15:35.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.947752+0000", 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024852799999999998 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.948499+0000", 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00074696199999999999 2024-09-21T23:15:35.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.952427+0000", 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039283060000000003 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.952591+0000", 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000164525 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.952712+0000", 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120229 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.952817+0000", 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105315 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1710 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.613 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.943880+0000", 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.5832244559999999, 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013061669999999999, 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1710 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943880+0000", 2024-09-21T23:15:35.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943880+0000", 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943881+0000", 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3370000000000001e-06 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943890+0000", 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8370000000000003e-06 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943915+0000", 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5239999999999999e-05 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.943931+0000", 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5857000000000001e-05 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949527+0000", 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055960519999999998 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949706+0000", 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000178603 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.950764+0000", 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010583680000000001 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.954421+0000", 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036562640000000002 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.956616+0000", 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0021951990000000001 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.956780+0000", 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016393000000000001 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.956942+0000", 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016198300000000001 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4541.0:1715 3.14 3:2f2e0b11:::rbd_object_map.10f1163c318d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:30.949082+0000", 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.5780229539999997, 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011322758, 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.619 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4541", 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1715 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949082+0000", 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949082+0000", 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949083+0000", 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.654e-06 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949093+0000", 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6830000000000003e-06 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949116+0000", 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2974000000000001e-05 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.949156+0000", 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0500999999999997e-05 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.954531+0000", 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005375169 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.954666+0000", 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401600000000001 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.955594+0000", 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00092844800000000001 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.957681+0000", 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002086858 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.960128+0000", 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024473749999999999 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.960292+0000", 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000163435 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:30.960404+0000", 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112645 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.624 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:186 3.14 3:2b1d47f7:::rbd_object_map.10f1dc9013:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.793355+0000", 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73374989199999996, 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.008993727, 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:35.625 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 186 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793355+0000", 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793355+0000", 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793355+0000", 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8699999999999995e-07 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793358+0000", 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6400000000000001e-06 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793362+0000", 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6550000000000003e-06 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793366+0000", 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.433e-06 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.796234+0000", 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028680590000000001 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.796299+0000", 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5154000000000007e-05 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.796638+0000", 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033870800000000001 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802128+0000", 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005490137 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802215+0000", 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7606000000000001e-05 2024-09-21T23:15:35.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802295+0000", 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9770000000000004e-05 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802348+0000", 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3078000000000002e-05 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:189 3.14 3:2b1d47f7:::rbd_object_map.10f1dc9013:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.793431+0000", 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73367326300000002, 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090744880000000003, 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 189 2024-09-21T23:15:35.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793431+0000", 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793431+0000", 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793432+0000", 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.96e-07 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793434+0000", 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3910000000000001e-06 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793440+0000", 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.755e-06 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793444+0000", 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4750000000000004e-06 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.797064+0000", 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036200569999999999 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.797131+0000", 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6501999999999994e-05 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.797595+0000", 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000464186 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802132+0000", 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045374100000000004 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802419+0000", 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028668600000000003 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802468+0000", 2024-09-21T23:15:35.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9484000000000003e-05 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802506+0000", 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7246000000000002e-05 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.636 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:196 3.14 3:2b1d47f7:::rbd_object_map.10f1dc9013:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.793518+0000", 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73358648900000001, 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091805050000000003, 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:35.637 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 196 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793518+0000", 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793518+0000", 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793518+0000", 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793522+0000", 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2179999999999999e-06 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793526+0000", 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7620000000000003e-06 2024-09-21T23:15:35.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793531+0000", 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6999999999999999e-06 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.798074+0000", 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045434029999999997 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.798143+0000", 2024-09-21T23:15:35.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9097999999999995e-05 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.798411+0000", 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026795999999999998 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802135+0000", 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037235039999999999 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802587+0000", 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00045234600000000001 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802652+0000", 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5074999999999996e-05 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802698+0000", 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6119999999999999e-05 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.642 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:201 3.14 3:2b1d47f7:::rbd_object_map.10f1dc9013:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.793587+0000", 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73351706000000005, 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0093483699999999999, 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:35.643 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 201 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793587+0000", 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793587+0000", 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793588+0000", 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9700000000000003e-07 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793590+0000", 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7640000000000001e-06 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793595+0000", 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8799999999999999e-06 2024-09-21T23:15:35.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.793599+0000", 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.304e-06 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.798994+0000", 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053952959999999999 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.799062+0000", 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8331999999999996e-05 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.799330+0000", 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026771299999999999 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802145+0000", 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028154180000000001 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802861+0000", 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00071600699999999995 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802904+0000", 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3069999999999999e-05 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.802936+0000", 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1288999999999998e-05 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:235 3.14 3:2b1d47f7:::rbd_object_map.10f1dc9013:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.794815+0000", 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.73228988299999997, 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013007711, 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:35.649 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 235 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.794815+0000", 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.794815+0000", 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:35.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.794817+0000", 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5900000000000002e-06 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.794833+0000", 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5396999999999998e-05 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.794866+0000", 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3237e-05 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.794889+0000", 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3003e-05 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.803278+0000", 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0083891929999999997 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.803393+0000", 2024-09-21T23:15:35.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114882 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.803986+0000", 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059298399999999996 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.805268+0000", 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001281955 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.807568+0000", 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023004670000000001 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.807704+0000", 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000135542 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.807822+0000", 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000118461 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.654 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.655 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.655 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:35.655 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:35.655 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:35.658 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:35.658 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:35.656+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:35.757 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:35.756+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:35.825 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:35.825 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:35.839 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:35.857 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:35.856+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:15:35.956 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:35.956 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:35.956 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:35.956 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:35.956 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:35.958 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:35.951+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:35.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:36.040 INFO:teuthology.orchestra.run.smithi170.stdout:1582 write 0x2012800 thru 0x20144b6 (0x1cb7 bytes) 2024-09-21T23:15:36.047 INFO:teuthology.orchestra.run.smithi170.stdout:1583 trunc from 0x2014600 to 0x3ff7a00 2024-09-21T23:15:36.063 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:36.059+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:36.087 INFO:teuthology.orchestra.run.smithi170.stdout:1585 write 0x41a8400 thru 0x41af55c (0x715d bytes) 2024-09-21T23:15:36.090 INFO:teuthology.orchestra.run.smithi170.stdout:1586 read 0x3a59800 thru 0x3a659ef (0xc1f0 bytes) 2024-09-21T23:15:36.121 INFO:teuthology.orchestra.run.smithi170.stdout:1590 write 0x6263400 thru 0x626b2c1 (0x7ec2 bytes) 2024-09-21T23:15:36.151 INFO:teuthology.orchestra.run.smithi170.stdout:1591 write 0x76a0e00 thru 0x76a89cd (0x7bce bytes) 2024-09-21T23:15:36.153 INFO:teuthology.orchestra.run.smithi170.stdout:1592 write 0x6da5800 thru 0x6da7264 (0x1a65 bytes) 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:276 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:36.154 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.201396572, 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014002473, 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 276 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:36.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949281+0000", 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949282+0000", 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.26e-06 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949286+0000", 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9230000000000004e-06 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949289+0000", 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0170000000000001e-06 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963078+0000", 2024-09-21T23:15:36.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013788804999999999 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963169+0000", 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1114999999999998e-05 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963283+0000", 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011409 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:277 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.201384088000001, 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014347459999999999, 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 277 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.159 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949293+0000", 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949294+0000", 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6199999999999999e-07 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949295+0000", 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2580000000000001e-06 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949299+0000", 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9199999999999997e-06 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949302+0000", 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9859999999999999e-06 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963339+0000", 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014037304 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963466+0000", 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012664600000000001 2024-09-21T23:15:36.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963641+0000", 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017508399999999999 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:278 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.201371694999999, 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014599439000000001, 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 278 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949306+0000", 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949307+0000", 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.139e-06 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949311+0000", 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0230000000000003e-06 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949314+0000", 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3139999999999999e-06 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963697+0000", 2024-09-21T23:15:36.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014382544000000001 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963785+0000", 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7587e-05 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963905+0000", 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012056 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:279 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.201358843000001, 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014999721000000001, 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 279 2024-09-21T23:15:36.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949318+0000", 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949319+0000", 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0600000000000001e-07 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949320+0000", 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.313e-06 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949324+0000", 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9570000000000004e-06 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949327+0000", 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.963958+0000", 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014630341 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964127+0000", 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016937300000000001 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964318+0000", 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019130699999999999 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:280 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.201325008000001, 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015213915, 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 280 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949352+0000", 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949353+0000", 2024-09-21T23:15:36.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949361+0000", 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7709999999999995e-06 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949367+0000", 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.003e-06 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.949371+0000", 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4000000000000001e-06 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964367+0000", 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014995746000000001 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964460+0000", 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3565000000000003e-05 2024-09-21T23:15:36.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.964566+0000", 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106067 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:350 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.183950729999999, 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013944434, 2024-09-21T23:15:36.177 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 350 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:36.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966727+0000", 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.65e-07 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966728+0000", 2024-09-21T23:15:36.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3069999999999999e-06 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966732+0000", 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9040000000000002e-06 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966735+0000", 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7819999999999998e-06 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980479+0000", 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013744528000000001 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980562+0000", 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2218999999999993e-05 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980671+0000", 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109429 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.181 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:351 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.183938627, 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014095582000000001, 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 351 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966739+0000", 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9799999999999999e-07 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966740+0000", 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1429999999999999e-06 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966744+0000", 2024-09-21T23:15:36.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5329999999999999e-06 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966747+0000", 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.117e-06 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980710+0000", 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013963074000000001 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980762+0000", 2024-09-21T23:15:36.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1779e-05 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980834+0000", 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2538000000000002e-05 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:352 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.183899057000001, 2024-09-21T23:15:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014313115, 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 352 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966778+0000", 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966779+0000", 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.22e-07 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966783+0000", 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8099999999999997e-06 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966788+0000", 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.758e-06 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966791+0000", 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1489999999999998e-06 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980882+0000", 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014090965 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.980969+0000", 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6222000000000002e-05 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981091+0000", 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000122889 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.190 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:353 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.183881548999999, 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014443615, 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 353 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966796+0000", 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.72e-07 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966802+0000", 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9070000000000004e-06 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966806+0000", 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1200000000000004e-06 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.966813+0000", 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5239999999999997e-06 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981114+0000", 2024-09-21T23:15:36.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014301385 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981166+0000", 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2021999999999998e-05 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:07.981239+0000", 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3385000000000003e-05 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:974 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 520192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.003338706000001, 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013957262999999999, 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 974 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147339+0000", 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9400000000000001e-07 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147340+0000", 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2589999999999999e-06 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147344+0000", 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1899999999999997e-06 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.147348+0000", 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161133+0000", 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013784597000000001 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161205+0000", 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2692000000000001e-05 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.161296+0000", 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0550000000000005e-05 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2367 3.16 3:6d83b35b:::rbd_object_map.10f16a447f06:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.637327596999999, 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015679904000000001, 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2367 2024-09-21T23:15:36.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513350+0000", 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513351+0000", 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513355+0000", 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5929999999999999e-06 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513372+0000", 2024-09-21T23:15:36.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7062999999999999e-05 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.513386+0000", 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3293e-05 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523249+0000", 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.009863251 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.523437+0000", 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018793 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.524029+0000", 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059259600000000003 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.525598+0000", 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001569017 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528769+0000", 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003170754 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.528893+0000", 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012397100000000001 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.529030+0000", 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013660899999999999 2024-09-21T23:15:36.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2583 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.596065312, 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013835072, 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2583 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554612+0000", 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1e-07 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554613+0000", 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.212e-06 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554617+0000", 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1240000000000001e-06 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554619+0000", 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6680000000000001e-06 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568239+0000", 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013620098000000001 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568331+0000", 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1255999999999995e-05 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568447+0000", 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011650399999999999 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2584 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:36.209 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.596054828, 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014048454, 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2584 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554623+0000", 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4999999999999999e-07 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554624+0000", 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.119e-06 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554627+0000", 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1200000000000002e-06 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554630+0000", 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7219999999999999e-06 2024-09-21T23:15:36.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568490+0000", 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013860268 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568570+0000", 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9903999999999999e-05 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568671+0000", 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101071 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2585 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 385024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.596044443, 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014283607, 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.214 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2585 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:36.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554633+0000", 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.77e-07 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554634+0000", 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.144e-06 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554638+0000", 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2440000000000002e-06 2024-09-21T23:15:36.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554640+0000", 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6869999999999999e-06 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568717+0000", 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014077118 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568802+0000", 2024-09-21T23:15:36.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4079000000000006e-05 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568917+0000", 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011495800000000001 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4405.0:2586 3.13 3:c8d03ab9:::rbd_data.10f16b870c42.0000000000000000:3 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:36.218 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.596020418999998, 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014464238000000001, 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4405", 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/1606768455", 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2586 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554657+0000", 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.28e-07 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554658+0000", 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.282e-06 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554662+0000", 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5470000000000001e-06 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.554665+0000", 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.568947+0000", 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01428159 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569026+0000", 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8950999999999997e-05 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:08.569121+0000", 2024-09-21T23:15:36.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5173000000000005e-05 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:388 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836097+0000", 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.314580238, 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015437466, 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 388 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836097+0000", 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836097+0000", 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836105+0000", 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5769999999999996e-06 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836114+0000", 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3109999999999995e-06 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836157+0000", 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2574000000000003e-05 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836192+0000", 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5027999999999997e-05 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.844648+0000", 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084561900000000006 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.844791+0000", 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014363099999999999 2024-09-21T23:15:36.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.845384+0000", 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000592133 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851166+0000", 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0057823789999999998 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851355+0000", 2024-09-21T23:15:36.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000188922 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851436+0000", 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1289e-05 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851535+0000", 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8431999999999995e-05 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:389 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836223+0000", 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.314453854, 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015457797000000001, 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 389 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.229 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836223+0000", 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836223+0000", 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836225+0000", 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8530000000000001e-06 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836234+0000", 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5739999999999996e-06 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836258+0000", 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3884e-05 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836297+0000", 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9453999999999999e-05 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.846771+0000", 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010473779000000001 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.846915+0000", 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144034 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.847985+0000", 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010695800000000001 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851333+0000", 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033487790000000001 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851553+0000", 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021979599999999999 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851625+0000", 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1449e-05 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851681+0000", 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6614999999999999e-05 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:390 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836358+0000", 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.314319347, 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020077536999999999, 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 390 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836358+0000", 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836358+0000", 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836376+0000", 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7594000000000001e-05 2024-09-21T23:15:36.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836386+0000", 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0193e-05 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836413+0000", 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7364000000000001e-05 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836429+0000", 2024-09-21T23:15:36.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5716999999999999e-05 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.849139+0000", 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012709651000000001 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.849374+0000", 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000235317 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:36.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.850308+0000", 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00093457200000000005 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856120+0000", 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0058112010000000002 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856241+0000", 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121669 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856318+0000", 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6699000000000005e-05 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856436+0000", 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011756000000000001 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.240 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:391 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836491+0000", 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.31418667, 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020212988000000001, 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 391 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836491+0000", 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836491+0000", 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836492+0000", 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0899999999999999e-06 2024-09-21T23:15:36.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836509+0000", 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6807e-05 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836531+0000", 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2240000000000001e-05 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836546+0000", 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5540999999999999e-05 2024-09-21T23:15:36.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851776+0000", 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015229397 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.851926+0000", 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014989100000000001 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.852867+0000", 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094158600000000001 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856221+0000", 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033537390000000001 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856466+0000", 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024535300000000002 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856628+0000", 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016166499999999999 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.856704+0000", 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5679000000000005e-05 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4477.0:392 3.5 3:a6a92186:::rbd_object_map.10f156ded9d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:22.836600+0000", 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.314076974000001, 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021550758, 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:36.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4477", 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3952092295", 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 392 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836600+0000", 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:36.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836600+0000", 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836602+0000", 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-06 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836610+0000", 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1599999999999998e-06 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:36.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836630+0000", 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0540999999999999e-05 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.836646+0000", 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.575e-05 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.854151+0000", 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017505007 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:36.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.854371+0000", 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000220306 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.855290+0000", 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00091819799999999995 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.857943+0000", 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026538030000000002 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.857966+0000", 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2251000000000001e-05 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.858047+0000", 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1852000000000002e-05 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:22.858151+0000", 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010367299999999999 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.252 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:36.252 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:36.252 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:36.253 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:36.247+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:1596 write 0x5ca0400 thru 0x5cab7d9 (0xb3da bytes) 2024-09-21T23:15:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:1597 write 0x4143e00 thru 0x41444c5 (0x6c6 bytes) 2024-09-21T23:15:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:1600 write 0x9eda000 thru 0x9ee12c0 (0x72c1 bytes) 2024-09-21T23:15:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:1602 write 0x568000 thru 0x5769e1 (0xe9e2 bytes) 2024-09-21T23:15:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:1605 trunc from 0x9ee1400 to 0x45a3200 2024-09-21T23:15:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:1606 read 0x495a00 thru 0x499eab (0x44ac bytes) 2024-09-21T23:15:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:1607 trunc from 0x45a3200 to 0xa4a6e00 2024-09-21T23:15:36.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:36.266 INFO:teuthology.orchestra.run.smithi170.stdout:1609 read 0x7475c00 thru 0x747dfd5 (0x83d6 bytes) 2024-09-21T23:15:36.268 INFO:teuthology.orchestra.run.smithi170.stdout:1610 punch from 0x87c4c00 to 0x87c5c00, (0x1000 bytes) 2024-09-21T23:15:36.269 INFO:teuthology.orchestra.run.smithi170.stdout:1614 write 0x5f15a00 thru 0x5f2520f (0xf810 bytes) 2024-09-21T23:15:36.269 INFO:teuthology.orchestra.run.smithi170.stdout:1618 write 0x36fce00 thru 0x37011fb (0x43fc bytes) 2024-09-21T23:15:36.269 INFO:teuthology.orchestra.run.smithi170.stdout:1621 write 0x8234e00 thru 0x8237b67 (0x2d68 bytes) 2024-09-21T23:15:36.270 INFO:teuthology.orchestra.run.smithi170.stdout:1622 write 0xa0bc000 thru 0xa0c3206 (0x7207 bytes) 2024-09-21T23:15:36.287 INFO:teuthology.orchestra.run.smithi170.stdout:1623 trunc from 0xa4a6e00 to 0xd9e00 2024-09-21T23:15:36.357 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:36.356+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:36.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:36.491 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:36.487+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:36.492 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:36.492 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:36.493 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1625 write 0xeb10800 thru 0xeb15b6c (0x536d bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1627 write 0x74c4c00 thru 0x74c4fd5 (0x3d6 bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1630 punch from 0xdfa4600 to 0xdfac800, (0x8200 bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1631 write 0xa195a00 thru 0xa1a140b (0xba0c bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1633 punch from 0x6859e00 to 0x685ce00, (0x3000 bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1641 punch from 0x16fc200 to 0x16ff000, (0x2e00 bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1647 write 0x2de6000 thru 0x2df489a (0xe89b bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1648 punch from 0x6c0400 to 0x6cc400, (0xc000 bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1650 read 0xd103000 thru 0xd11031c (0xd31d bytes) 2024-09-21T23:15:36.494 INFO:teuthology.orchestra.run.smithi170.stdout:1653 write 0x2f13a00 thru 0x2f152ff (0x1900 bytes) 2024-09-21T23:15:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:1654 trunc from 0xeb15c00 to 0x7c53c00 2024-09-21T23:15:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:1655 read 0x759de00 thru 0x75ac268 (0xe469 bytes) 2024-09-21T23:15:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:1656 read 0x361a00 thru 0x3677f3 (0x5df4 bytes) 2024-09-21T23:15:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:1657 read 0x64fa200 thru 0x6505aa6 (0xb8a7 bytes) 2024-09-21T23:15:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:1660 read 0x50b8c00 thru 0x50c0f77 (0x8378 bytes) 2024-09-21T23:15:36.523 INFO:teuthology.orchestra.run.smithi170.stdout:1666 write 0xd606a00 thru 0xd6070a4 (0x6a5 bytes) 2024-09-21T23:15:36.526 INFO:teuthology.orchestra.run.smithi170.stdout:1667 flatten 2024-09-21T23:15:36.556 INFO:teuthology.orchestra.run.smithi170.stdout:1670 write 0x13aa200 thru 0x13aa611 (0x412 bytes) 2024-09-21T23:15:36.586 INFO:teuthology.orchestra.run.smithi170.stdout:1672 write 0xed08200 thru 0xed0bf30 (0x3d31 bytes) 2024-09-21T23:15:36.590 INFO:teuthology.orchestra.run.smithi170.stdout:1674 punch from 0xe462a00 to 0xe466e00, (0x4400 bytes) 2024-09-21T23:15:36.591 INFO:teuthology.orchestra.run.smithi170.stdout:1676 write 0xbe75600 thru 0xbe7e494 (0x8e95 bytes) 2024-09-21T23:15:36.591 INFO:teuthology.orchestra.run.smithi170.stdout:1677 punch from 0x68c0000 to 0x68c9400, (0x9400 bytes) 2024-09-21T23:15:36.591 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:36.588+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:36.592 INFO:teuthology.orchestra.run.smithi170.stdout:1680 read 0xcb07600 thru 0xcb11645 (0xa046 bytes) 2024-09-21T23:15:36.592 INFO:teuthology.orchestra.run.smithi170.stdout:1682 read 0x3ce3c00 thru 0x3ce8837 (0x4c38 bytes) 2024-09-21T23:15:36.592 INFO:teuthology.orchestra.run.smithi170.stdout:1683 write 0xcbb000 thru 0xcc46d6 (0x96d7 bytes) 2024-09-21T23:15:36.614 INFO:teuthology.orchestra.run.smithi170.stdout:1684 trunc from 0xed0c000 to 0x2469e00 2024-09-21T23:15:36.671 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:36.671 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:36.671 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:36.671 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:36.672 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:36.687 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:36.692 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:36.687+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:36.692 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:36.706 INFO:teuthology.orchestra.run.smithi170.stdout:1686 write 0x9d5d000 thru 0x9d665ca (0x95cb bytes) 2024-09-21T23:15:36.711 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:36.711 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:36.712 INFO:teuthology.orchestra.run.smithi170.stdout:1689 trunc from 0x9d66600 to 0xaf70600 2024-09-21T23:15:36.731 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:36.731 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:36.739 INFO:teuthology.orchestra.run.smithi170.stdout:1692 punch from 0x288a00 to 0x297200, (0xe800 bytes) 2024-09-21T23:15:36.741 INFO:teuthology.orchestra.run.smithi170.stdout:1698 read 0x53e3400 thru 0x53ec5e2 (0x91e3 bytes) 2024-09-21T23:15:36.741 INFO:teuthology.orchestra.run.smithi170.stdout:1699 write 0x3717e00 thru 0x371eddb (0x6fdc bytes) 2024-09-21T23:15:36.755 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:36.755 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:36.757 INFO:teuthology.orchestra.run.smithi170.stdout:1702 trunc from 0xaf70600 to 0x2f78600 2024-09-21T23:15:36.777 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:36.792 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:36.787+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:36.800 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:36.800 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:36.803 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:36.803 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:36.817 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:36.817 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:36.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:36.839 INFO:teuthology.orchestra.run.smithi170.stdout:1705 write 0xad24200 thru 0xad3177e (0xd57f bytes) 2024-09-21T23:15:36.844 INFO:teuthology.orchestra.run.smithi170.stdout:1706 trunc from 0xad31800 to 0xbc4a200 2024-09-21T23:15:36.881 INFO:teuthology.orchestra.run.smithi170.stdout:1707 read 0xaf65a00 thru 0xaf69bb0 (0x41b1 bytes) 2024-09-21T23:15:36.884 INFO:teuthology.orchestra.run.smithi170.stdout:1708 read 0x9ae3400 thru 0x9ae91c0 (0x5dc1 bytes) 2024-09-21T23:15:36.885 INFO:teuthology.orchestra.run.smithi170.stdout:1709 read 0x5055200 thru 0x5060ee4 (0xbce5 bytes) 2024-09-21T23:15:36.885 INFO:teuthology.orchestra.run.smithi170.stdout:1710 punch from 0x77c7a00 to 0x77d6800, (0xee00 bytes) 2024-09-21T23:15:36.900 INFO:teuthology.orchestra.run.smithi170.stdout:1711 trunc from 0xbc4a200 to 0x4b76400 2024-09-21T23:15:36.920 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:36.920+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:36.957 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5] out_osds: [3, 1] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-21T23:15:36.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:15:36.958 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T23:15:36.958 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:15:36.958 INFO:teuthology.orchestra.run.smithi170.stdout:1712 write 0x968f600 thru 0x9690d29 (0x172a bytes) 2024-09-21T23:15:36.961 INFO:teuthology.orchestra.run.smithi170.stdout:1713 read 0x4f0dc00 thru 0x4f18dce (0xb1cf bytes) 2024-09-21T23:15:36.964 INFO:teuthology.orchestra.run.smithi170.stdout:1716 trunc from 0x9690e00 to 0x7f30000 2024-09-21T23:15:37.004 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:37.004+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:37.006 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:37.006 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:37.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:37.031 INFO:teuthology.orchestra.run.smithi170.stdout:1718 write 0x908ca00 thru 0x9097ac9 (0xb0ca bytes) 2024-09-21T23:15:37.071 INFO:teuthology.orchestra.run.smithi170.stdout:1719 write 0xc94ea00 thru 0xc958603 (0x9c04 bytes) 2024-09-21T23:15:37.090 INFO:teuthology.orchestra.run.smithi170.stdout:1720 trunc from 0xc958800 to 0x48c7a00 2024-09-21T23:15:37.120 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:37.120+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:37.133 INFO:teuthology.orchestra.run.smithi170.stdout:1722 punch from 0x164b000 to 0x1655800, (0xa800 bytes) 2024-09-21T23:15:37.135 INFO:teuthology.orchestra.run.smithi170.stdout:1723 read 0x470c000 thru 0x470ed88 (0x2d89 bytes) 2024-09-21T23:15:37.135 INFO:teuthology.orchestra.run.smithi170.stdout:truncating to largest ever: 0xec98000 2024-09-21T23:15:37.160 INFO:teuthology.orchestra.run.smithi170.stdout:1724 trunc from 0x48c7a00 to 0xec98000 2024-09-21T23:15:37.181 INFO:teuthology.orchestra.run.smithi170.stdout:1726 write 0xd52ac00 thru 0xd52e10f (0x3510 bytes) 2024-09-21T23:15:37.184 INFO:teuthology.orchestra.run.smithi170.stdout:1727 write 0x61e6c00 thru 0x61ef8f8 (0x8cf9 bytes) 2024-09-21T23:15:37.184 INFO:teuthology.orchestra.run.smithi170.stdout:1728 write 0x8a92400 thru 0x8a955d2 (0x31d3 bytes) 2024-09-21T23:15:37.184 INFO:teuthology.orchestra.run.smithi170.stdout:1731 write 0xb4c4800 thru 0xb4c6c94 (0x2495 bytes) 2024-09-21T23:15:37.184 INFO:teuthology.orchestra.run.smithi170.stdout:1733 read 0xab47800 thru 0xab5685f (0xf060 bytes) 2024-09-21T23:15:37.184 INFO:teuthology.orchestra.run.smithi170.stdout:1734 read 0x763f800 thru 0x7647f9a (0x879b bytes) 2024-09-21T23:15:37.184 INFO:teuthology.orchestra.run.smithi170.stdout:1736 write 0x9e46600 thru 0x9e480b1 (0x1ab2 bytes) 2024-09-21T23:15:37.184 INFO:teuthology.orchestra.run.smithi170.stdout:1737 punch from 0x5b25800 to 0x5b35000, (0xf800 bytes) 2024-09-21T23:15:37.185 INFO:teuthology.orchestra.run.smithi170.stdout:1741 write 0xd03b800 thru 0xd047eb8 (0xc6b9 bytes) 2024-09-21T23:15:37.185 INFO:teuthology.orchestra.run.smithi170.stdout:1742 read 0xa15c000 thru 0xa163e45 (0x7e46 bytes) 2024-09-21T23:15:37.185 INFO:teuthology.orchestra.run.smithi170.stdout:1745 read 0x4473200 thru 0x447ee0a (0xbc0b bytes) 2024-09-21T23:15:37.186 INFO:teuthology.orchestra.run.smithi170.stdout:1746 punch from 0x8133800 to 0x813a800, (0x7000 bytes) 2024-09-21T23:15:37.186 INFO:teuthology.orchestra.run.smithi170.stdout:1747 read 0x753d800 thru 0x75448f6 (0x70f7 bytes) 2024-09-21T23:15:37.186 INFO:teuthology.orchestra.run.smithi170.stdout:1748 read 0x1e64800 thru 0x1e6de59 (0x965a bytes) 2024-09-21T23:15:37.188 INFO:teuthology.orchestra.run.smithi170.stdout:1752 read 0x2cfc000 thru 0x2d099e8 (0xd9e9 bytes) 2024-09-21T23:15:37.191 INFO:teuthology.orchestra.run.smithi170.stdout:1754 punch from 0x772da00 to 0x772f600, (0x1c00 bytes) 2024-09-21T23:15:37.191 INFO:teuthology.orchestra.run.smithi170.stdout:1758 write 0x84b6800 thru 0x84bc153 (0x5954 bytes) 2024-09-21T23:15:37.191 INFO:teuthology.orchestra.run.smithi170.stdout:1759 read 0x610a200 thru 0x61121a7 (0x7fa8 bytes) 2024-09-21T23:15:37.191 INFO:teuthology.orchestra.run.smithi170.stdout:1760 write 0x2528c00 thru 0x25387f2 (0xfbf3 bytes) 2024-09-21T23:15:37.192 INFO:teuthology.orchestra.run.smithi170.stdout:1761 read 0xea59a00 thru 0xea65917 (0xbf18 bytes) 2024-09-21T23:15:37.195 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:37.191+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:37.198 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:37.198 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:37.212 INFO:teuthology.orchestra.run.smithi170.stdout:1762 trunc from 0xec98000 to 0x2fd5400 2024-09-21T23:15:37.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 45, 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "309803bf-3365-47ac-8bf6-365125beceb1", 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-21T23:14:32.028556+0000", 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-21T23:15:35.955053+0000", 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-21T23:15:11.366587+0000", 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-21T23:15:31.924617+0000", 2024-09-21T23:15:37.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5801984, 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "noscrub", 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 5, 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-21T23:15:37.280 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 6, 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "jewel", 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "squid", 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:44.926374+0000", 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:15:37.281 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-21T23:15:37.282 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "15", 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-21T23:15:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:15:37.284 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:15:37.285 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 5.880000114440918, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 5.880000114440918, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.17000000178813934, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:15:37.286 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:49.598603+0000", 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:15:37.287 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:15:37.288 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "19", 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 2, 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 19, 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:15:37.289 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:15:37.290 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:15:37.291 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 3, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 3, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.5, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.5, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.5, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:15:37.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:59.367341+0000", 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:15:37.293 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:15:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "41", 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 8, 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 41, 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:15:37.295 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:15:37.296 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:15:37.297 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2.2599999904632568, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2.2599999904632568, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.5, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "a24df615-4443-4f27-888f-d1e4e7affbb3", 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 0, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 28, 2024-09-21T23:15:37.299 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 31, 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.301 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.302 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-21T23:15:37.303 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 91058490 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/91058490", 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/91058490", 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/91058490", 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/91058490", 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "exists" 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:15:37.304 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d6765c2a-98b2-4a2b-8d8c-f3963575eb64", 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 43, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/2712042978", 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/2712042978", 2024-09-21T23:15:37.309 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2712042978", 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2712042978", 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "9753272a-58a9-4366-87d7-550c49134023", 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:15:37.310 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 44, 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.311 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.313 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-21T23:15:37.314 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/1166517858", 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/1166517858", 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/1166517858", 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/1166517858", 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "24d987f9-f2cd-4ccd-9dc8-fea64467d92d", 2024-09-21T23:15:37.315 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 21, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6809", 2024-09-21T23:15:37.316 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6811", 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6814", 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6816", 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6822", 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6823", 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6818", 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.319 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6821", 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6811/1039392120", 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6816/1039392120", 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6823/1039392120", 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6821/1039392120", 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.320 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "c1c30a49-7f26-4264-a1a9-160dc552953d", 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 44, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.321 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6808", 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6810", 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.322 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6812", 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6813", 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6819", 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6820", 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6815", 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6817", 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6810/2764095640", 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6813/2764095640", 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6820/2764095640", 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6817/2764095640", 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f24e9341-be4e-4be2-b180-7ebd3f705e9b", 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 44, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6800", 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6801", 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6802", 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6803", 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6806", 2024-09-21T23:15:37.328 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6807", 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6804", 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6805", 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6801/1732973431", 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6803/1732973431", 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6807/1732973431", 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6805/1732973431", 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:15:37.330 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.468749+0000", 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 31 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.044837+0000", 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:15:37.332 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.310970+0000", 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.241402+0000", 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.333 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.877818+0000", 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:15:37.334 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.434052+0000", 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [ 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "1.0", 2024-09-21T23:15:37.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.0", 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.1", 2024-09-21T23:15:37.336 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.2", 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: 5, 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.4", 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-09-21T23:15:37.337 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "2.6", 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.4", 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-09-21T23:15:37.338 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.6", 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.b", 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: 4, 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.12", 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: 2, 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: 1 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-21T23:15:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:15:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-21T23:15:37.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:37.342 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:37.343 INFO:teuthology.orchestra.run.smithi170.stdout:1764 write 0x72da800 thru 0x72dea3a (0x423b bytes) 2024-09-21T23:15:37.343 INFO:teuthology.orchestra.run.smithi170.stdout:1769 punch from 0x67d1e00 to 0x67dbc00, (0x9e00 bytes) 2024-09-21T23:15:37.343 INFO:teuthology.orchestra.run.smithi170.stdout:1770 trunc from 0x72dec00 to 0x9f53800 2024-09-21T23:15:37.344 INFO:tasks.thrashosds.thrasher:j is {'epoch': 45, 'fsid': '309803bf-3365-47ac-8bf6-365125beceb1', 'created': '2024-09-21T23:14:32.028556+0000', 'modified': '2024-09-21T23:15:35.955053+0000', 'last_up_change': '2024-09-21T23:15:11.366587+0000', 'last_in_change': '2024-09-21T23:15:31.924617+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:14:44.926374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:14:49.598603+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T23:14:59.367341+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 8, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.259999990463257, 'score_stable': 2.259999990463257, 'optimal_score': 0.5, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a24df615-4443-4f27-888f-d1e4e7affbb3', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 28, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 91058490}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 91058490}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 91058490}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 91058490}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 91058490}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 91058490}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 91058490}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 91058490}]}, 'public_addr': '172.21.15.53:6817/91058490', 'cluster_addr': '172.21.15.53:6819/91058490', 'heartbeat_back_addr': '172.21.15.53:6823/91058490', 'heartbeat_front_addr': '172.21.15.53:6821/91058490', 'state': ['exists']}, {'osd': 1, 'uuid': 'd6765c2a-98b2-4a2b-8d8c-f3963575eb64', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2712042978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2712042978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2712042978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2712042978}]}, 'public_addr': '172.21.15.53:6809/2712042978', 'cluster_addr': '172.21.15.53:6811/2712042978', 'heartbeat_back_addr': '172.21.15.53:6815/2712042978', 'heartbeat_front_addr': '172.21.15.53:6813/2712042978', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9753272a-58a9-4366-87d7-550c49134023', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 1166517858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 1166517858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 1166517858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 1166517858}]}, 'public_addr': '172.21.15.53:6801/1166517858', 'cluster_addr': '172.21.15.53:6803/1166517858', 'heartbeat_back_addr': '172.21.15.53:6807/1166517858', 'heartbeat_front_addr': '172.21.15.53:6805/1166517858', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '24d987f9-f2cd-4ccd-9dc8-fea64467d92d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6809', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 1039392120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6816', 'nonce': 1039392120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6823', 'nonce': 1039392120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 1039392120}]}, 'public_addr': '172.21.15.144:6811/1039392120', 'cluster_addr': '172.21.15.144:6816/1039392120', 'heartbeat_back_addr': '172.21.15.144:6823/1039392120', 'heartbeat_front_addr': '172.21.15.144:6821/1039392120', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'c1c30a49-7f26-4264-a1a9-160dc552953d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6810', 'nonce': 2764095640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 2764095640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2764095640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6815', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2764095640}]}, 'public_addr': '172.21.15.144:6810/2764095640', 'cluster_addr': '172.21.15.144:6813/2764095640', 'heartbeat_back_addr': '172.21.15.144:6820/2764095640', 'heartbeat_front_addr': '172.21.15.144:6817/2764095640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f24e9341-be4e-4be2-b180-7ebd3f705e9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 1732973431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 1732973431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 1732973431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 1732973431}]}, 'public_addr': '172.21.15.144:6801/1732973431', 'cluster_addr': '172.21.15.144:6803/1732973431', 'heartbeat_back_addr': '172.21.15.144:6807/1732973431', 'heartbeat_front_addr': '172.21.15.144:6805/1732973431', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.468749+0000', 'dead_epoch': 31}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.044837+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.310970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.241402+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.877818+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.434052+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '1.0', 'osds': [2, 1]}, {'pgid': '2.0', 'osds': [5, 1]}, {'pgid': '2.1', 'osds': [2, 1]}, {'pgid': '2.2', 'osds': [5, 1]}, {'pgid': '2.4', 'osds': [4, 1]}, {'pgid': '2.6', 'osds': [4, 1]}, {'pgid': '3.4', 'osds': [2, 1]}, {'pgid': '3.6', 'osds': [4, 1]}, {'pgid': '3.b', 'osds': [4, 1]}, {'pgid': '3.12', 'osds': [2, 1]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:15:37.344 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T23:15:37.344 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:37.339+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:37.351 INFO:teuthology.orchestra.run.smithi170.stdout:1772 punch from 0x94c5000 to 0x94c8800, (0x3800 bytes) 2024-09-21T23:15:37.353 INFO:teuthology.orchestra.run.smithi170.stdout:1773 read 0x1c4cc00 thru 0x1c5c05b (0xf45c bytes) 2024-09-21T23:15:37.387 INFO:teuthology.orchestra.run.smithi170.stdout:1775 write 0xea04800 thru 0xea0d2f3 (0x8af4 bytes) 2024-09-21T23:15:37.389 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:37.389 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:37.389 INFO:teuthology.orchestra.run.smithi170.stdout:1776 read 0x5871400 thru 0x5878be4 (0x77e5 bytes) 2024-09-21T23:15:37.402 INFO:teuthology.orchestra.run.smithi170.stdout:1777 trunc from 0xea0d400 to 0x9f28c00 2024-09-21T23:15:37.404 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:37.445 INFO:teuthology.orchestra.run.smithi170.stdout:1778 read 0x4419800 thru 0x441b596 (0x1d97 bytes) 2024-09-21T23:15:37.451 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:37.447+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:37.475 INFO:teuthology.orchestra.run.smithi170.stdout:1780 write 0xc94da00 thru 0xc94f58b (0x1b8c bytes) 2024-09-21T23:15:37.477 INFO:teuthology.orchestra.run.smithi170.stdout:1784 read 0x116e400 thru 0x117a0a4 (0xbca5 bytes) 2024-09-21T23:15:37.481 INFO:teuthology.orchestra.run.smithi170.stdout:1785 punch from 0x1c61200 to 0x1c70000, (0xee00 bytes) 2024-09-21T23:15:37.482 INFO:teuthology.orchestra.run.smithi170.stdout:1788 read 0xc35a600 thru 0xc35dab0 (0x34b1 bytes) 2024-09-21T23:15:37.482 INFO:teuthology.orchestra.run.smithi170.stdout:1791 read 0x31f800 thru 0x32606d (0x686e bytes) 2024-09-21T23:15:37.486 INFO:teuthology.orchestra.run.smithi170.stdout:1794 read 0xb8eca00 thru 0xb8fb4a4 (0xeaa5 bytes) 2024-09-21T23:15:37.488 INFO:teuthology.orchestra.run.smithi170.stdout:1796 punch from 0x58f1e00 to 0x58f8000, (0x6200 bytes) 2024-09-21T23:15:37.488 INFO:teuthology.orchestra.run.smithi170.stdout:1797 punch from 0x5518000 to 0x5522000, (0xa000 bytes) 2024-09-21T23:15:37.507 INFO:teuthology.orchestra.run.smithi170.stdout:1800 trunc from 0xc94f600 to 0x4bb1e00 2024-09-21T23:15:37.534 INFO:teuthology.orchestra.run.smithi170.stdout:1801 punch from 0x113b600 to 0x1142a00, (0x7400 bytes) 2024-09-21T23:15:37.536 INFO:teuthology.orchestra.run.smithi170.stdout:1802 write 0x2970e00 thru 0x2978f13 (0x8114 bytes) 2024-09-21T23:15:37.539 INFO:teuthology.orchestra.run.smithi170.stdout:1804 trunc from 0x4bb1e00 to 0x375f800 2024-09-21T23:15:37.545 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:37.544+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:37.570 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:37.570 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:37.571 INFO:teuthology.orchestra.run.smithi170.stdout:1806 read 0x142b200 thru 0x143a89b (0xf69c bytes) 2024-09-21T23:15:37.576 INFO:teuthology.orchestra.run.smithi170.stdout:1807 punch from 0x5e0200 to 0x5e9400, (0x9200 bytes) 2024-09-21T23:15:37.576 INFO:teuthology.orchestra.run.smithi170.stdout:1809 read 0x13b5c00 thru 0x13be16e (0x856f bytes) 2024-09-21T23:15:37.585 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:37.611 INFO:teuthology.orchestra.run.smithi170.stdout:1811 write 0xa6c4800 thru 0xa6cb183 (0x6984 bytes) 2024-09-21T23:15:37.613 INFO:teuthology.orchestra.run.smithi170.stdout:1812 write 0x93a4000 thru 0x93ab096 (0x7097 bytes) 2024-09-21T23:15:37.613 INFO:teuthology.orchestra.run.smithi170.stdout:1813 punch from 0x58a2400 to 0x58a5c00, (0x3800 bytes) 2024-09-21T23:15:37.613 INFO:teuthology.orchestra.run.smithi170.stdout:1817 write 0x1e89400 thru 0x1e98405 (0xf006 bytes) 2024-09-21T23:15:37.614 INFO:teuthology.orchestra.run.smithi170.stdout:1818 punch from 0x6afae00 to 0x6b07e00, (0xd000 bytes) 2024-09-21T23:15:37.615 INFO:teuthology.orchestra.run.smithi170.stdout:1820 read 0x6473600 thru 0x647fa27 (0xc428 bytes) 2024-09-21T23:15:37.615 INFO:teuthology.orchestra.run.smithi170.stdout:1824 read 0x6d2da00 thru 0x6d3aa89 (0xd08a bytes) 2024-09-21T23:15:37.615 INFO:teuthology.orchestra.run.smithi170.stdout:1825 read 0x89e6400 thru 0x89e9d80 (0x3981 bytes) 2024-09-21T23:15:37.625 INFO:teuthology.orchestra.run.smithi170.stdout:1826 trunc from 0xa6cb200 to 0x4155a00 2024-09-21T23:15:37.645 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:37.644+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:37.656 INFO:teuthology.orchestra.run.smithi170.stdout:truncating to largest ever: 0xee6a400 2024-09-21T23:15:37.683 INFO:teuthology.orchestra.run.smithi170.stdout:1827 trunc from 0x4155a00 to 0xee6a400 2024-09-21T23:15:37.701 INFO:teuthology.orchestra.run.smithi170.stdout:1829 write 0x3953800 thru 0x39605be (0xcdbf bytes) 2024-09-21T23:15:37.702 INFO:teuthology.orchestra.run.smithi170.stdout:1830 punch from 0x969800 to 0x979000, (0xf800 bytes) 2024-09-21T23:15:37.703 INFO:teuthology.orchestra.run.smithi170.stdout:1831 read 0x49d9c00 thru 0x49e474d (0xab4e bytes) 2024-09-21T23:15:37.707 INFO:teuthology.orchestra.run.smithi170.stdout:1832 read 0x37f800 thru 0x386196 (0x6997 bytes) 2024-09-21T23:15:37.710 INFO:teuthology.orchestra.run.smithi170.stdout:1833 read 0x3e68a00 thru 0x3e719c3 (0x8fc4 bytes) 2024-09-21T23:15:37.712 INFO:teuthology.orchestra.run.smithi170.stdout:1835 write 0x1c8aa00 thru 0x1c8c3d9 (0x19da bytes) 2024-09-21T23:15:37.712 INFO:teuthology.orchestra.run.smithi170.stdout:1838 read 0xa1a8c00 thru 0xa1af4f0 (0x68f1 bytes) 2024-09-21T23:15:37.713 INFO:teuthology.orchestra.run.smithi170.stdout:1839 write 0x5a4800 thru 0x5b11ec (0xc9ed bytes) 2024-09-21T23:15:37.714 INFO:teuthology.orchestra.run.smithi170.stdout:1840 write 0x5ff3400 thru 0x60033ec (0xffed bytes) 2024-09-21T23:15:37.714 INFO:teuthology.orchestra.run.smithi170.stdout:1842 write 0x43e3200 thru 0x43ec960 (0x9761 bytes) 2024-09-21T23:15:37.715 INFO:teuthology.orchestra.run.smithi170.stdout:1846 read 0x7cb9a00 thru 0x7cbc1a3 (0x27a4 bytes) 2024-09-21T23:15:37.715 INFO:teuthology.orchestra.run.smithi170.stdout:1847 write 0xc3bf600 thru 0xc3c734a (0x7d4b bytes) 2024-09-21T23:15:37.715 INFO:teuthology.orchestra.run.smithi170.stdout:1849 write 0x179cc00 thru 0x17a6a03 (0x9e04 bytes) 2024-09-21T23:15:37.742 INFO:teuthology.orchestra.run.smithi170.stdout:1850 trunc from 0xee6a400 to 0xf7c200 2024-09-21T23:15:37.746 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:37.739+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:37.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:37.826 INFO:teuthology.orchestra.run.smithi170.stdout:1854 write 0x48e4600 thru 0x48e8f9b (0x499c bytes) 2024-09-21T23:15:37.858 INFO:teuthology.orchestra.run.smithi170.stdout:1855 write 0x81f6600 thru 0x81fe1ec (0x7bed bytes) 2024-09-21T23:15:37.862 INFO:teuthology.orchestra.run.smithi170.stdout:1857 trunc from 0x81fe200 to 0x8ba7a00 2024-09-21T23:15:37.867 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:37.863+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:37.886 INFO:teuthology.orchestra.run.smithi170.stdout:1861 read 0x52e5800 thru 0x52ee2de (0x8adf bytes) 2024-09-21T23:15:37.909 INFO:teuthology.orchestra.run.smithi170.stdout:1862 write 0xa0b4200 thru 0xa0c3eed (0xfcee bytes) 2024-09-21T23:15:37.912 INFO:teuthology.orchestra.run.smithi170.stdout:1865 punch from 0x1082800 to 0x1090a00, (0xe200 bytes) 2024-09-21T23:15:37.913 INFO:teuthology.orchestra.run.smithi170.stdout:1866 write 0xd3ec00 thru 0xd418ce (0x2ccf bytes) 2024-09-21T23:15:37.914 INFO:teuthology.orchestra.run.smithi170.stdout:1867 clone 7 order 21 su 65536 sc 5 2024-09-21T23:15:37.949 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:37.948+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:37.958 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:37.959 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:37.973 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:37.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:15:38.049 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:38.048+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:38.140 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:38.141 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:38.150 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:38.143+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:38.155 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:38.250 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:38.247+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:38.332 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:38.332 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:38.347 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:38.352 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:38.348+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:38.453 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:38.447+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:38.517 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:38.518 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:38.532 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:38.554 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:38.552+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:38.655 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:38.651+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:38.670 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-21T23:15:38.696 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:38.696 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:38.711 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:38.755 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:38.751+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:38.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:38.857 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:38.856+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:38.878 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:38.878 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:38.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:38.942 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:38.957 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:38.956+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:38.960 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:38.976 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:38.977 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:38.993 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:15:39.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:39.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:39.013 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:39.013 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:39.026 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:39.027 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:39.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:39.046 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:39.046 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:39.049 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:39.049 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:39.092 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:39.092+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:39.144 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:39.144 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:39.144 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:39.144 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:39.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:39.160 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:39.156+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:39.261 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:39.255+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:39.328 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:39.328 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:39.328 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:39.328 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:39.328 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:39.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:39.362 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:39.355+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:39.390 INFO:teuthology.orchestra.run.smithi170.stdout:1869 write 0xcd5600 thru 0xce006a (0xaa6b bytes) 2024-09-21T23:15:39.403 INFO:teuthology.orchestra.run.smithi170.stdout:1870 read 0x8ca7e00 thru 0x8cb418a (0xc38b bytes) 2024-09-21T23:15:39.405 INFO:teuthology.orchestra.run.smithi170.stdout:1872 read 0x2fe3e00 thru 0x2ff061c (0xc81d bytes) 2024-09-21T23:15:39.407 INFO:teuthology.orchestra.run.smithi170.stdout:1873 read 0x493e000 thru 0x494de18 (0xfe19 bytes) 2024-09-21T23:15:39.409 INFO:teuthology.orchestra.run.smithi170.stdout:1874 read 0x831c400 thru 0x8325380 (0x8f81 bytes) 2024-09-21T23:15:39.412 INFO:teuthology.orchestra.run.smithi170.stdout:1876 read 0x5407e00 thru 0x541105b (0x925c bytes) 2024-09-21T23:15:39.413 INFO:teuthology.orchestra.run.smithi170.stdout:1877 punch from 0x6b32a00 to 0x6b33c00, (0x1200 bytes) 2024-09-21T23:15:39.462 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:39.455+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:39.472 INFO:teuthology.orchestra.run.smithi170.stdout:1878 read 0x19ed000 thru 0x19f33ea (0x63eb bytes) 2024-09-21T23:15:39.474 INFO:teuthology.orchestra.run.smithi170.stdout:1881 punch from 0x85e200 to 0x862800, (0x4600 bytes) 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.137694691, 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:39.486 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.487 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.487 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:39.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:39.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:39.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:39.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:39.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:39.493 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.37005233, 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.990742248, 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.863525236000001, 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.948761505, 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:15:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.493808738999999, 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.643782355999999, 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:15:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:15:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:15:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.643740938000001, 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:15:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.461655012000001, 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:15:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:15:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:15:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.319712669000001, 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:15:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.163846374, 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:15:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:15:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:15:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.001116097000001, 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:15:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:15:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.481690714000001, 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:15:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.481562238, 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:15:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:15:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.480600534000001, 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:15:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:15:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:15:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.479589323000001, 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.478507376, 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6019291179999993, 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:15:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:15:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6015895479999998, 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:15:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:15:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:596 3.12 3:4d026f6e:::rbd_data.10f1dc9013.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5318745370000002, 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078799430000000004, 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 596 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951020+0000", 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-06 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951929+0000", 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090880699999999997 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951965+0000", 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5734e-05 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951982+0000", 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6806000000000001e-05 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952135+0000", 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015325899999999999 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952560+0000", 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042476500000000002 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.953181+0000", 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062147299999999999 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.957005+0000", 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038238 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958342+0000", 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001336622 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958557+0000", 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021509599999999999 2024-09-21T23:15:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958897+0000", 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034014100000000002 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:39.602 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:39.604 INFO:teuthology.orchestra.run.smithi170.stdout:1882 write 0x246ac00 thru 0x2477069 (0xc46a bytes) 2024-09-21T23:15:39.604 INFO:teuthology.orchestra.run.smithi170.stdout:1883 punch from 0x3fe1600 to 0x3fe2000, (0xa00 bytes) 2024-09-21T23:15:39.604 INFO:teuthology.orchestra.run.smithi170.stdout:1884 read 0x8d72000 thru 0x8d7c4c2 (0xa4c3 bytes) 2024-09-21T23:15:39.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:39.605 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:39.599+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:39.637 INFO:teuthology.orchestra.run.smithi170.stdout:1885 trunc from 0xa0c4000 to 0x4416000 2024-09-21T23:15:39.704 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:39.699+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:39.724 INFO:teuthology.orchestra.run.smithi170.stdout:1889 trunc from 0x4416000 to 0xaf23400 2024-09-21T23:15:39.742 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:39.742 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:39.742 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:39.742 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:39.747 INFO:teuthology.orchestra.run.smithi170.stdout:1890 write 0xb36400 thru 0xb3781b (0x141c bytes) 2024-09-21T23:15:39.754 INFO:teuthology.orchestra.run.smithi170.stdout:1893 read 0x87caa00 thru 0x87ce535 (0x3b36 bytes) 2024-09-21T23:15:39.754 INFO:teuthology.orchestra.run.smithi170.stdout:1895 read 0x755da00 thru 0x75653e4 (0x79e5 bytes) 2024-09-21T23:15:39.756 INFO:teuthology.orchestra.run.smithi170.stdout:1896 write 0x4e9aa00 thru 0x4ea5589 (0xab8a bytes) 2024-09-21T23:15:39.756 INFO:teuthology.orchestra.run.smithi170.stdout:1900 read 0x7b36600 thru 0x7b423d5 (0xbdd6 bytes) 2024-09-21T23:15:39.756 INFO:teuthology.orchestra.run.smithi170.stdout:1903 write 0x4af2800 thru 0x4afe855 (0xc056 bytes) 2024-09-21T23:15:39.757 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:39.805 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:39.799+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:39.829 INFO:teuthology.orchestra.run.smithi170.stdout:1905 write 0xe87e000 thru 0xe8817aa (0x37ab bytes) 2024-09-21T23:15:39.832 INFO:teuthology.orchestra.run.smithi170.stdout:1907 read 0x29c0000 thru 0x29c5ec6 (0x5ec7 bytes) 2024-09-21T23:15:39.833 INFO:teuthology.orchestra.run.smithi170.stdout:1909 write 0x1b2d800 thru 0x1b3cff7 (0xf7f8 bytes) 2024-09-21T23:15:39.835 INFO:teuthology.orchestra.run.smithi170.stdout:1917 punch from 0xe7e6800 to 0xe7f5c00, (0xf400 bytes) 2024-09-21T23:15:39.840 INFO:teuthology.orchestra.run.smithi170.stdout:1918 punch from 0xd264e00 to 0xd272600, (0xd800 bytes) 2024-09-21T23:15:39.840 INFO:teuthology.orchestra.run.smithi170.stdout:1919 write 0x6f26200 thru 0x6f341c6 (0xdfc7 bytes) 2024-09-21T23:15:39.865 INFO:teuthology.orchestra.run.smithi170.stdout:1922 trunc from 0xe881800 to 0x3e87200 2024-09-21T23:15:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:39.905 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:39.899+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:39.907 INFO:teuthology.orchestra.run.smithi170.stdout:1923 read 0xcb4c00 thru 0xcbebe8 (0x9fe9 bytes) 2024-09-21T23:15:39.910 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:39.911 INFO:teuthology.orchestra.run.smithi170.stdout:1925 read 0x25bb000 thru 0x25bdcb4 (0x2cb5 bytes) 2024-09-21T23:15:39.942 INFO:teuthology.orchestra.run.smithi170.stdout:1927 write 0x88ac600 thru 0x88b84a7 (0xbea8 bytes) 2024-09-21T23:15:39.972 INFO:teuthology.orchestra.run.smithi170.stdout:1930 write 0x9663400 thru 0x967007c (0xcc7d bytes) 2024-09-21T23:15:40.007 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:40.004+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:40.007 INFO:teuthology.orchestra.run.smithi170.stdout:1935 write 0xb71ec00 thru 0xb7249b3 (0x5db4 bytes) 2024-09-21T23:15:40.039 INFO:teuthology.orchestra.run.smithi170.stdout:1937 write 0xc93d000 thru 0xc94b537 (0xe538 bytes) 2024-09-21T23:15:40.040 INFO:teuthology.orchestra.run.smithi170.stdout:1939 read 0x241a200 thru 0x2427c1f (0xda20 bytes) 2024-09-21T23:15:40.047 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:40.047 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:40.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:40.047 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:40.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.047 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.047 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.118171029, 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:40.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:40.052 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1171501819999996, 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.053 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:40.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:15:40.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:15:40.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1170905089999996, 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1170005329999997, 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:15:40.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1169506230000001, 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.068 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:15:40.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1168944830000003, 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:15:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:15:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1168407, 2024-09-21T23:15:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:15:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1168084240000002, 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:15:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1167781440000004, 2024-09-21T23:15:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1167484879999998, 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:15:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:15:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:15:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1167185210000001, 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1166406919999998, 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:15:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:15:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:15:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:15:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1165929849999996, 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:15:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1165627889999996, 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:15:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:15:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1165328719999996, 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1165030720000004, 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.116473139, 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1164081760000002, 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:15:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:15:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:15:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1163777000000001, 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:15:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1163485360000003, 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:15:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:15:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:40.150 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:40.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:40.154 INFO:teuthology.orchestra.run.smithi170.stdout:1941 trunc from 0xc94b600 to 0x776a400 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1942 trunc from 0x776a400 to 0xc178c00 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1943 write 0xae8d800 thru 0xae964cd (0x8cce bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1948 read 0x1653c00 thru 0x165e2da (0xa6db bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1949 read 0x63f8400 thru 0x64015fc (0x91fd bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1955 read 0x2a8fe00 thru 0x2a9b768 (0xb969 bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1956 read 0x6047000 thru 0x6052d56 (0xbd57 bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1957 read 0x81cb800 thru 0x81db663 (0xfe64 bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1958 read 0x4409a00 thru 0x441165f (0x7c60 bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1959 write 0x511bc00 thru 0x512ab22 (0xef23 bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1960 read 0xa3b6200 thru 0xa3c0f38 (0xad39 bytes) 2024-09-21T23:15:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:1961 punch from 0xbeddc00 to 0xbee2200, (0x4600 bytes) 2024-09-21T23:15:40.156 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:40.147+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:40.167 INFO:teuthology.orchestra.run.smithi170.stdout:1965 write 0xc474a00 thru 0xc47c46f (0x7a70 bytes) 2024-09-21T23:15:40.170 INFO:teuthology.orchestra.run.smithi170.stdout:1966 write 0x6331a00 thru 0x6338ea9 (0x74aa bytes) 2024-09-21T23:15:40.170 INFO:teuthology.orchestra.run.smithi170.stdout:1967 write 0x24cf000 thru 0x24d005e (0x105f bytes) 2024-09-21T23:15:40.172 INFO:teuthology.orchestra.run.smithi170.stdout:1972 read 0x8ff1000 thru 0x8ff88d2 (0x78d3 bytes) 2024-09-21T23:15:40.180 INFO:teuthology.orchestra.run.smithi170.stdout:1975 trunc from 0xc47c600 to 0xea7ce00 2024-09-21T23:15:40.210 INFO:teuthology.orchestra.run.smithi170.stdout:1977 clone 8 order 23 su 2097152 sc 12 2024-09-21T23:15:40.253 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:40.247+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:40.294 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:40.295 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:40.309 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:40.395 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:40.391+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:40.454 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:40.447+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:40.475 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:40.475 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:40.476 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:40.476 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:40.476 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:40.489 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:40.556 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:40.556+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.433072871, 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.920667588000001, 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.920602684999999, 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.920456246000001, 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.920387130999998, 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.919774806, 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.919570401000001, 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.919527615, 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:40.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.528569327, 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:40.707 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.528482816999997, 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.528405927999998, 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.528201088000003, 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.528112806000003, 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.527981978, 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.527936324999999, 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.527897897999999, 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.527768436000002, 2024-09-21T23:15:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.527746794000002, 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.527726749999999, 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 53.527642757999999, 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:40.779 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:40.779 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:40.651+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:40.780 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:40.823 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:40.819+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:40.883 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:40.884+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:40.949 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:40.949 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:40.949 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:40.949 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:40.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:40.984 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:40.984+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:41.016 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:15:41.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:41.085 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:41.079+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:41.091 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:41.091 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:41.091 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:41.091 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:41.091 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:41.106 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:41.139 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone7 from 0xea7ce00 (overlap 0x3e87200) to 0x640a00 2024-09-21T23:15:41.185 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:41.184+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:41.205 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:41.224 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:41.224 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:41.246 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:41.246 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:41.268 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:41.268 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:41.285 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:41.279+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:41.288 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:41.288 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:41.307 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:41.308 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:41.310 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:41.311 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:41.319 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:41.319 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:41.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:41.418 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:41.415+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:41.489 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:41.488+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:41.499 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:41.499 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:41.514 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:41.590 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:41.588+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:41.686 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:41.687 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:41.691 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:41.687+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:41.701 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:41.770 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-21T23:15:41.793 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:41.792+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:41.887 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:41.887 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:41.894 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:41.892+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:41.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:41.995 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:41.991+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:42.001 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:15:42.065 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:42.065 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:42.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:42.095 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:42.091+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:42.196 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:42.191+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:42.274 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:42.274 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:42.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:42.296 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:42.291+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:42.345 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5] out_osds: [3, 1] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-21T23:15:42.345 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:15:42.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:15:42.397 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:42.391+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:42.426 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:42.426 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:42.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:42.498 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:42.495+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:42.585 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:42.585 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:42.600 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:42.596+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:42.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:15:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:15:42.700 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:42.700+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:42.719 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-21T23:15:42.000626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103076177,"num_objects":190,"num_object_clones":24,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":25,"num_read":56743,"num_read_kb":168937,"num_write":7610,"num_write_kb":29912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":1154754,"num_keys_recovered":25,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6319,"ondisk_log_size":6319,"up":60,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":148792,"kb_used_data":7340,"kb_used_omap":6,"kb_used_meta":141305,"kb_avail":377338568,"statfs":{"total":386547056640,"available":386394693632,"internally_reserved":0,"allocated":7516160,"data_stored":5898284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6380,"internal_metadata":144697108},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48371843,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":8490,"num_read_kb":34712,"num_write":960,"num_write_kb":8924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.146652"},"pg_stats":[{"pgid":"3.1f","version":"47'101","reported_seq":203,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.871473+0000","last_change":"2024-09-21T23:15:11.740491+0000","last_active":"2024-09-21T23:15:39.871473+0000","last_peered":"2024-09-21T23:15:39.871473+0000","last_clean":"2024-09-21T23:15:08.357378+0000","last_became_active":"2024-09-21T23:15:11.740491+0000","last_became_peered":"2024-09-21T23:15:11.740491+0000","last_unstale":"2024-09-21T23:15:39.871473+0000","last_undegraded":"2024-09-21T23:15:11.739054+0000","last_fullsized":"2024-09-21T23:15:11.738940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:06.652964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":2748496,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":202,"num_read_kb":216,"num_write":128,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"47'120","reported_seq":355,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:38.994983+0000","last_change":"2024-09-21T23:15:35.968961+0000","last_active":"2024-09-21T23:15:38.994983+0000","last_peered":"2024-09-21T23:15:38.994983+0000","last_clean":"2024-09-21T23:15:38.994983+0000","last_became_active":"2024-09-21T23:15:35.968136+0000","last_became_peered":"2024-09-21T23:15:35.968136+0000","last_unstale":"2024-09-21T23:15:38.994983+0000","last_undegraded":"2024-09-21T23:15:38.994983+0000","last_fullsized":"2024-09-21T23:15:38.994983+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:05:15.060922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":1997824,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":933,"num_write":120,"num_write_kb":603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"45'47","reported_seq":609,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.853173+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:15:39.853173+0000","last_peered":"2024-09-21T23:15:39.853173+0000","last_clean":"2024-09-21T23:15:39.853173+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:15:39.853173+0000","last_undegraded":"2024-09-21T23:15:39.853173+0000","last_fullsized":"2024-09-21T23:15:39.853173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:09:22.621155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":2722304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":501,"num_read_kb":3022,"num_write":50,"num_write_kb":672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"47'151","reported_seq":783,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.860404+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:15:39.860404+0000","last_peered":"2024-09-21T23:15:39.860404+0000","last_clean":"2024-09-21T23:15:39.860404+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:15:39.860404+0000","last_undegraded":"2024-09-21T23:15:39.860404+0000","last_fullsized":"2024-09-21T23:15:39.860404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:49.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":5454912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":596,"num_read_kb":4547,"num_write":153,"num_write_kb":2822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"45'214","reported_seq":1121,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.873717+0000","last_change":"2024-09-21T23:15:11.742173+0000","last_active":"2024-09-21T23:15:39.873717+0000","last_peered":"2024-09-21T23:15:39.873717+0000","last_clean":"2024-09-21T23:15:08.357275+0000","last_became_active":"2024-09-21T23:15:11.742173+0000","last_became_peered":"2024-09-21T23:15:11.742173+0000","last_unstale":"2024-09-21T23:15:39.873717+0000","last_undegraded":"2024-09-21T23:15:11.740235+0000","last_fullsized":"2024-09-21T23:15:11.739985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:17:57.071287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":4764190,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1832,"num_read_kb":4475,"num_write":238,"num_write_kb":719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"46'245","reported_seq":4024,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.826903+0000","last_change":"2024-09-21T23:15:35.969265+0000","last_active":"2024-09-21T23:15:39.826903+0000","last_peered":"2024-09-21T23:15:39.826903+0000","last_clean":"2024-09-21T23:15:39.826903+0000","last_became_active":"2024-09-21T23:15:35.967793+0000","last_became_peered":"2024-09-21T23:15:35.967793+0000","last_unstale":"2024-09-21T23:15:39.826903+0000","last_undegraded":"2024-09-21T23:15:39.826903+0000","last_fullsized":"2024-09-21T23:15:39.826903+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:14.527629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":2521619,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":4840,"num_read_kb":11788,"num_write":181,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":39443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"45'79","reported_seq":479,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.871989+0000","last_change":"2024-09-21T23:15:32.983007+0000","last_active":"2024-09-21T23:15:39.871989+0000","last_peered":"2024-09-21T23:15:39.871989+0000","last_clean":"2024-09-21T23:15:39.871989+0000","last_became_active":"2024-09-21T23:15:32.948191+0000","last_became_peered":"2024-09-21T23:15:32.948191+0000","last_unstale":"2024-09-21T23:15:39.871989+0000","last_undegraded":"2024-09-21T23:15:39.871989+0000","last_fullsized":"2024-09-21T23:15:39.871989+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:07.848138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":2506752,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":303,"num_read_kb":2716,"num_write":80,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"45'78","reported_seq":1208,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:38.994640+0000","last_change":"2024-09-21T23:15:13.071510+0000","last_active":"2024-09-21T23:15:38.994640+0000","last_peered":"2024-09-21T23:15:38.994640+0000","last_clean":"2024-09-21T23:15:10.369468+0000","last_became_active":"2024-09-21T23:15:11.740045+0000","last_became_peered":"2024-09-21T23:15:11.740045+0000","last_unstale":"2024-09-21T23:15:38.994640+0000","last_undegraded":"2024-09-21T23:15:12.997527+0000","last_fullsized":"2024-09-21T23:15:11.738605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:33:56.587767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":2243072,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":1061,"num_read_kb":7892,"num_write":77,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":5}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"47'115","reported_seq":285,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.755764+0000","last_change":"2024-09-21T23:15:12.744639+0000","last_active":"2024-09-21T23:15:39.755764+0000","last_peered":"2024-09-21T23:15:39.755764+0000","last_clean":"2024-09-21T23:15:09.361363+0000","last_became_active":"2024-09-21T23:15:12.744639+0000","last_became_peered":"2024-09-21T23:15:12.744639+0000","last_unstale":"2024-09-21T23:15:39.755764+0000","last_undegraded":"2024-09-21T23:15:12.742574+0000","last_fullsized":"2024-09-21T23:15:12.742439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:26.780052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":2286080,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":106,"num_read_kb":657,"num_write":112,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":6}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"47'216","reported_seq":1339,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.757386+0000","last_change":"2024-09-21T23:15:35.965248+0000","last_active":"2024-09-21T23:15:39.757386+0000","last_peered":"2024-09-21T23:15:39.757386+0000","last_clean":"2024-09-21T23:15:39.757386+0000","last_became_active":"2024-09-21T23:15:35.963512+0000","last_became_peered":"2024-09-21T23:15:35.963512+0000","last_unstale":"2024-09-21T23:15:39.757386+0000","last_undegraded":"2024-09-21T23:15:39.757386+0000","last_fullsized":"2024-09-21T23:15:39.757386+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:40:44.722653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":5211810,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1747,"num_read_kb":5001,"num_write":329,"num_write_kb":1855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":94370,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"45'97","reported_seq":1668,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.465042+0000","last_change":"2024-09-21T23:15:14.751494+0000","last_active":"2024-09-21T23:15:39.465042+0000","last_peered":"2024-09-21T23:15:39.465042+0000","last_clean":"2024-09-21T23:15:08.790498+0000","last_became_active":"2024-09-21T23:15:11.743088+0000","last_became_peered":"2024-09-21T23:15:11.743088+0000","last_unstale":"2024-09-21T23:15:39.465042+0000","last_undegraded":"2024-09-21T23:15:14.648933+0000","last_fullsized":"2024-09-21T23:15:11.741363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:23.080202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4694528,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1519,"num_read_kb":9112,"num_write":97,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":3}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"47'453","reported_seq":1479,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.545934+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:15:39.545934+0000","last_peered":"2024-09-21T23:15:39.545934+0000","last_clean":"2024-09-21T23:15:39.545934+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:15:39.545934+0000","last_undegraded":"2024-09-21T23:15:39.545934+0000","last_fullsized":"2024-09-21T23:15:39.545934+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:19:18.337499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294177,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2746,"num_read_kb":9092,"num_write":700,"num_write_kb":1231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.395019+0000","last_change":"2024-09-21T23:15:06.350164+0000","last_active":"2024-09-21T23:15:39.395019+0000","last_peered":"2024-09-21T23:15:39.395019+0000","last_clean":"2024-09-21T23:15:39.395019+0000","last_became_active":"2024-09-21T23:15:06.349271+0000","last_became_peered":"2024-09-21T23:15:06.349271+0000","last_unstale":"2024-09-21T23:15:39.395019+0000","last_undegraded":"2024-09-21T23:15:39.395019+0000","last_fullsized":"2024-09-21T23:15:39.395019+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:56.638782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"45'123","reported_seq":317,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:39.395154+0000","last_change":"2024-09-21T23:15:35.966252+0000","last_active":"2024-09-21T23:15:39.395154+0000","last_peered":"2024-09-21T23:15:39.395154+0000","last_clean":"2024-09-21T23:15:39.395154+0000","last_became_active":"2024-09-21T23:15:35.964302+0000","last_became_peered":"2024-09-21T23:15:35.964302+0000","last_unstale":"2024-09-21T23:15:39.395154+0000","last_undegraded":"2024-09-21T23:15:39.395154+0000","last_fullsized":"2024-09-21T23:15:39.395154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:19:25.885116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":253952,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":612,"num_write":123,"num_write_kb":523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,4","objects":2}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:39.394931+0000","last_change":"2024-09-21T23:15:33.957450+0000","last_active":"2024-09-21T23:15:39.394931+0000","last_peered":"2024-09-21T23:15:39.394931+0000","last_clean":"2024-09-21T23:15:39.394931+0000","last_became_active":"2024-09-21T23:15:33.956367+0000","last_became_peered":"2024-09-21T23:15:33.956367+0000","last_unstale":"2024-09-21T23:15:39.394931+0000","last_undegraded":"2024-09-21T23:15:39.394931+0000","last_fullsized":"2024-09-21T23:15:39.394931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:44.313635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"45'1423","reported_seq":2874,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:38.994894+0000","last_change":"2024-09-21T23:15:35.969593+0000","last_active":"2024-09-21T23:15:38.994894+0000","last_peered":"2024-09-21T23:15:38.994894+0000","last_clean":"2024-09-21T23:15:38.994894+0000","last_became_active":"2024-09-21T23:15:35.967156+0000","last_became_peered":"2024-09-21T23:15:35.967156+0000","last_unstale":"2024-09-21T23:15:38.994894+0000","last_undegraded":"2024-09-21T23:15:38.994894+0000","last_fullsized":"2024-09-21T23:15:38.994894+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1423,"log_dups_size":0,"ondisk_log_size":1423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:33:07.732570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":5300608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8422,"num_read_kb":13886,"num_write":2581,"num_write_kb":3468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":247168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":47,"state":"active+undersized","last_fresh":"2024-09-21T23:15:39.394241+0000","last_change":"2024-09-21T23:15:11.743860+0000","last_active":"2024-09-21T23:15:39.394241+0000","last_peered":"2024-09-21T23:15:39.394241+0000","last_clean":"2024-09-21T23:14:50.229607+0000","last_became_active":"2024-09-21T23:15:11.743860+0000","last_became_peered":"2024-09-21T23:15:11.743860+0000","last_unstale":"2024-09-21T23:15:39.394241+0000","last_undegraded":"2024-09-21T23:15:39.394241+0000","last_fullsized":"2024-09-21T23:15:11.741411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:54:20.545029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"40'70","reported_seq":189,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:38.994952+0000","last_change":"2024-09-21T23:15:33.946002+0000","last_active":"2024-09-21T23:15:38.994952+0000","last_peered":"2024-09-21T23:15:38.994952+0000","last_clean":"2024-09-21T23:15:38.994952+0000","last_became_active":"2024-09-21T23:15:33.945362+0000","last_became_peered":"2024-09-21T23:15:33.945362+0000","last_unstale":"2024-09-21T23:15:38.994952+0000","last_undegraded":"2024-09-21T23:15:38.994952+0000","last_fullsized":"2024-09-21T23:15:38.994952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:08.922709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":233472,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":46,"num_read_kb":374,"num_write":69,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:39.394027+0000","last_change":"2024-09-21T23:15:33.946139+0000","last_active":"2024-09-21T23:15:39.394027+0000","last_peered":"2024-09-21T23:15:39.394027+0000","last_clean":"2024-09-21T23:15:39.394027+0000","last_became_active":"2024-09-21T23:15:33.944897+0000","last_became_peered":"2024-09-21T23:15:33.944897+0000","last_unstale":"2024-09-21T23:15:39.394027+0000","last_undegraded":"2024-09-21T23:15:39.394027+0000","last_fullsized":"2024-09-21T23:15:39.394027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:08:45.179781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"47'213","reported_seq":716,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.856270+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:15:39.856270+0000","last_peered":"2024-09-21T23:15:39.856270+0000","last_clean":"2024-09-21T23:15:39.856270+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:15:39.856270+0000","last_undegraded":"2024-09-21T23:15:39.856270+0000","last_fullsized":"2024-09-21T23:15:39.856270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:53:13.227202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":8345886,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":942,"num_read_kb":3548,"num_write":283,"num_write_kb":1207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"47'124","reported_seq":1475,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.394378+0000","last_change":"2024-09-21T23:15:14.751409+0000","last_active":"2024-09-21T23:15:39.394378+0000","last_peered":"2024-09-21T23:15:39.394378+0000","last_clean":"2024-09-21T23:15:10.359509+0000","last_became_active":"2024-09-21T23:15:11.744261+0000","last_became_peered":"2024-09-21T23:15:11.744261+0000","last_unstale":"2024-09-21T23:15:39.394378+0000","last_undegraded":"2024-09-21T23:15:14.671830+0000","last_fullsized":"2024-09-21T23:15:11.740789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:35:40.879833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":1743872,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1281,"num_read_kb":8326,"num_write":123,"num_write_kb":558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":4}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":80,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:39.400421+0000","last_change":"2024-09-21T23:15:33.946743+0000","last_active":"2024-09-21T23:15:39.400421+0000","last_peered":"2024-09-21T23:15:39.400421+0000","last_clean":"2024-09-21T23:15:39.400421+0000","last_became_active":"2024-09-21T23:15:33.945318+0000","last_became_peered":"2024-09-21T23:15:33.945318+0000","last_unstale":"2024-09-21T23:15:39.400421+0000","last_undegraded":"2024-09-21T23:15:39.400421+0000","last_fullsized":"2024-09-21T23:15:39.400421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:43.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"47'126","reported_seq":650,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.856390+0000","last_change":"2024-09-21T23:15:35.969320+0000","last_active":"2024-09-21T23:15:39.856390+0000","last_peered":"2024-09-21T23:15:39.856390+0000","last_clean":"2024-09-21T23:15:39.856390+0000","last_became_active":"2024-09-21T23:15:35.968596+0000","last_became_peered":"2024-09-21T23:15:35.968596+0000","last_unstale":"2024-09-21T23:15:39.856390+0000","last_undegraded":"2024-09-21T23:15:39.856390+0000","last_fullsized":"2024-09-21T23:15:39.856390+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:26.896137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":2288179,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":406,"num_read_kb":3420,"num_write":124,"num_write_kb":485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":29200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:38.994747+0000","last_change":"2024-09-21T23:15:33.949165+0000","last_active":"2024-09-21T23:15:38.994747+0000","last_peered":"2024-09-21T23:15:38.994747+0000","last_clean":"2024-09-21T23:15:38.994747+0000","last_became_active":"2024-09-21T23:15:33.945922+0000","last_became_peered":"2024-09-21T23:15:33.945922+0000","last_unstale":"2024-09-21T23:15:38.994747+0000","last_undegraded":"2024-09-21T23:15:38.994747+0000","last_fullsized":"2024-09-21T23:15:38.994747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:29:49.233361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"47'152","reported_seq":726,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.872355+0000","last_change":"2024-09-21T23:15:11.742147+0000","last_active":"2024-09-21T23:15:39.872355+0000","last_peered":"2024-09-21T23:15:39.872355+0000","last_clean":"2024-09-21T23:15:08.563705+0000","last_became_active":"2024-09-21T23:15:11.742147+0000","last_became_peered":"2024-09-21T23:15:11.742147+0000","last_unstale":"2024-09-21T23:15:39.872355+0000","last_undegraded":"2024-09-21T23:15:11.739928+0000","last_fullsized":"2024-09-21T23:15:11.739661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:39.940968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":3493888,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":515,"num_read_kb":3057,"num_write":154,"num_write_kb":1838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:39.400178+0000","last_change":"2024-09-21T23:15:33.969515+0000","last_active":"2024-09-21T23:15:39.400178+0000","last_peered":"2024-09-21T23:15:39.400178+0000","last_clean":"2024-09-21T23:15:39.400178+0000","last_became_active":"2024-09-21T23:15:33.968382+0000","last_became_peered":"2024-09-21T23:15:33.968382+0000","last_unstale":"2024-09-21T23:15:39.400178+0000","last_undegraded":"2024-09-21T23:15:39.400178+0000","last_fullsized":"2024-09-21T23:15:39.400178+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:11.105738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"47'111","reported_seq":400,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.776417+0000","last_change":"2024-09-21T23:15:33.016643+0000","last_active":"2024-09-21T23:15:39.776417+0000","last_peered":"2024-09-21T23:15:39.776417+0000","last_clean":"2024-09-21T23:15:39.776417+0000","last_became_active":"2024-09-21T23:15:32.955127+0000","last_became_peered":"2024-09-21T23:15:32.955127+0000","last_unstale":"2024-09-21T23:15:39.776417+0000","last_undegraded":"2024-09-21T23:15:39.776417+0000","last_fullsized":"2024-09-21T23:15:39.776417+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:04.023671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":2121789,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":182,"num_read_kb":998,"num_write":111,"num_write_kb":522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":10301,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.400122+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:15:39.400122+0000","last_peered":"2024-09-21T23:15:39.400122+0000","last_clean":"2024-09-21T23:15:39.400122+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:15:39.400122+0000","last_undegraded":"2024-09-21T23:15:39.400122+0000","last_fullsized":"2024-09-21T23:15:39.400122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:38.994602+0000","last_change":"2024-09-21T23:15:35.966790+0000","last_active":"2024-09-21T23:15:38.994602+0000","last_peered":"2024-09-21T23:15:38.994602+0000","last_clean":"2024-09-21T23:15:38.994602+0000","last_became_active":"2024-09-21T23:15:35.965835+0000","last_became_peered":"2024-09-21T23:15:35.965835+0000","last_unstale":"2024-09-21T23:15:38.994602+0000","last_undegraded":"2024-09-21T23:15:38.994602+0000","last_fullsized":"2024-09-21T23:15:38.994602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:06:54.446732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"47'476","reported_seq":9049,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.872056+0000","last_change":"2024-09-21T23:15:35.968122+0000","last_active":"2024-09-21T23:15:39.872056+0000","last_peered":"2024-09-21T23:15:39.872056+0000","last_clean":"2024-09-21T23:15:39.872056+0000","last_became_active":"2024-09-21T23:15:35.967154+0000","last_became_peered":"2024-09-21T23:15:35.967154+0000","last_unstale":"2024-09-21T23:15:39.872056+0000","last_undegraded":"2024-09-21T23:15:39.872056+0000","last_fullsized":"2024-09-21T23:15:39.872056+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:52.536925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":2231824,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":9816,"num_read_kb":11053,"num_write":275,"num_write_kb":1125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":15888,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"45'236","reported_seq":2146,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.393899+0000","last_change":"2024-09-21T23:15:03.325366+0000","last_active":"2024-09-21T23:15:39.393899+0000","last_peered":"2024-09-21T23:15:39.393899+0000","last_clean":"2024-09-21T23:15:39.393899+0000","last_became_active":"2024-09-21T23:15:01.329082+0000","last_became_peered":"2024-09-21T23:15:01.329082+0000","last_unstale":"2024-09-21T23:15:39.393899+0000","last_undegraded":"2024-09-21T23:15:39.393899+0000","last_fullsized":"2024-09-21T23:15:39.393899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:26.161017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":1988608,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":3214,"num_read_kb":3618,"num_write":174,"num_write_kb":630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"46'96","reported_seq":356,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.393396+0000","last_change":"2024-09-21T23:15:33.025560+0000","last_active":"2024-09-21T23:15:39.393396+0000","last_peered":"2024-09-21T23:15:39.393396+0000","last_clean":"2024-09-21T23:15:39.393396+0000","last_became_active":"2024-09-21T23:15:32.949370+0000","last_became_peered":"2024-09-21T23:15:32.949370+0000","last_unstale":"2024-09-21T23:15:39.393396+0000","last_undegraded":"2024-09-21T23:15:39.393396+0000","last_fullsized":"2024-09-21T23:15:39.393396+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:13:18.375941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1333248,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":161,"num_read_kb":897,"num_write":95,"num_write_kb":503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":8192,"num_keys_recovered":3,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"47'112","reported_seq":1146,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:39.445399+0000","last_change":"2024-09-21T23:15:35.966303+0000","last_active":"2024-09-21T23:15:39.445399+0000","last_peered":"2024-09-21T23:15:39.445399+0000","last_clean":"2024-09-21T23:15:39.445399+0000","last_became_active":"2024-09-21T23:15:35.964609+0000","last_became_peered":"2024-09-21T23:15:35.964609+0000","last_unstale":"2024-09-21T23:15:39.445399+0000","last_undegraded":"2024-09-21T23:15:39.445399+0000","last_fullsized":"2024-09-21T23:15:39.445399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:12.951725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":1384463,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":975,"num_read_kb":8174,"num_write":112,"num_write_kb":572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":94208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,4","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"47'171","reported_seq":416,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.858406+0000","last_change":"2024-09-21T23:15:12.743649+0000","last_active":"2024-09-21T23:15:39.858406+0000","last_peered":"2024-09-21T23:15:39.858406+0000","last_clean":"2024-09-21T23:15:10.367359+0000","last_became_active":"2024-09-21T23:15:12.743649+0000","last_became_peered":"2024-09-21T23:15:12.743649+0000","last_unstale":"2024-09-21T23:15:39.858406+0000","last_undegraded":"2024-09-21T23:15:12.742456+0000","last_fullsized":"2024-09-21T23:15:12.742371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:31:28.706513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":3379757,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":295,"num_read_kb":854,"num_write":182,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":8}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"45'107","reported_seq":2221,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.847938+0000","last_change":"2024-09-21T23:15:06.357124+0000","last_active":"2024-09-21T23:15:39.847938+0000","last_peered":"2024-09-21T23:15:39.847938+0000","last_clean":"2024-09-21T23:15:39.847938+0000","last_became_active":"2024-09-21T23:15:06.356745+0000","last_became_peered":"2024-09-21T23:15:06.356745+0000","last_unstale":"2024-09-21T23:15:39.847938+0000","last_undegraded":"2024-09-21T23:15:39.847938+0000","last_fullsized":"2024-09-21T23:15:39.847938+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:56:18.698772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3740160,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2062,"num_read_kb":17394,"num_write":108,"num_write_kb":853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"47'98","reported_seq":410,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.828787+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:15:39.828787+0000","last_peered":"2024-09-21T23:15:39.828787+0000","last_clean":"2024-09-21T23:15:39.828787+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:15:39.828787+0000","last_undegraded":"2024-09-21T23:15:39.828787+0000","last_fullsized":"2024-09-21T23:15:39.828787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:15:36.870827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":2166784,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":375,"num_read_kb":1593,"num_write":92,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"47'110","reported_seq":1073,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.859576+0000","last_change":"2024-09-21T23:15:23.818610+0000","last_active":"2024-09-21T23:15:39.859576+0000","last_peered":"2024-09-21T23:15:39.859576+0000","last_clean":"2024-09-21T23:15:10.367429+0000","last_became_active":"2024-09-21T23:15:12.744651+0000","last_became_peered":"2024-09-21T23:15:12.744651+0000","last_unstale":"2024-09-21T23:15:39.859576+0000","last_undegraded":"2024-09-21T23:15:23.585152+0000","last_fullsized":"2024-09-21T23:15:12.742533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:02.687356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":4331520,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1210,"num_read_kb":3082,"num_write":94,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":4}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"47'142","reported_seq":1387,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.400679+0000","last_change":"2024-09-21T23:15:12.742870+0000","last_active":"2024-09-21T23:15:39.400679+0000","last_peered":"2024-09-21T23:15:39.400679+0000","last_clean":"2024-09-21T23:15:10.367211+0000","last_became_active":"2024-09-21T23:15:12.742870+0000","last_became_peered":"2024-09-21T23:15:12.742870+0000","last_unstale":"2024-09-21T23:15:39.400679+0000","last_undegraded":"2024-09-21T23:15:12.742124+0000","last_fullsized":"2024-09-21T23:15:12.742024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:50:17.062698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":3205632,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":1179,"num_read_kb":8103,"num_write":141,"num_write_kb":773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":6}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"47'81","reported_seq":250,"reported_epoch":47,"state":"active+undersized+degraded","last_fresh":"2024-09-21T23:15:39.754369+0000","last_change":"2024-09-21T23:15:11.742245+0000","last_active":"2024-09-21T23:15:39.754369+0000","last_peered":"2024-09-21T23:15:39.754369+0000","last_clean":"2024-09-21T23:15:10.359214+0000","last_became_active":"2024-09-21T23:15:11.742245+0000","last_became_peered":"2024-09-21T23:15:11.742245+0000","last_unstale":"2024-09-21T23:15:39.754369+0000","last_undegraded":"2024-09-21T23:15:11.740079+0000","last_fullsized":"2024-09-21T23:15:11.739829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:14:57.964369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3126272,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":110,"num_read_kb":592,"num_write":80,"num_write_kb":337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":4}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"47'199","reported_seq":1916,"reported_epoch":47,"state":"active+clean+remapped","last_fresh":"2024-09-21T23:15:39.875831+0000","last_change":"2024-09-21T23:15:35.967412+0000","last_active":"2024-09-21T23:15:39.875831+0000","last_peered":"2024-09-21T23:15:39.875831+0000","last_clean":"2024-09-21T23:15:39.875831+0000","last_became_active":"2024-09-21T23:15:35.965548+0000","last_became_peered":"2024-09-21T23:15:35.965548+0000","last_unstale":"2024-09-21T23:15:39.875831+0000","last_undegraded":"2024-09-21T23:15:39.875831+0000","last_fullsized":"2024-09-21T23:15:39.875831+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:37:45.882287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":4581888,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2793,"num_read_kb":5349,"num_write":151,"num_write_kb":851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":8192,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"45'113","reported_seq":7116,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T23:15:39.400279+0000","last_change":"2024-09-21T23:15:06.345498+0000","last_active":"2024-09-21T23:15:39.400279+0000","last_peered":"2024-09-21T23:15:39.400279+0000","last_clean":"2024-09-21T23:15:39.400279+0000","last_became_active":"2024-09-21T23:15:06.344491+0000","last_became_peered":"2024-09-21T23:15:06.344491+0000","last_unstale":"2024-09-21T23:15:39.400279+0000","last_undegraded":"2024-09-21T23:15:39.400279+0000","last_fullsized":"2024-09-21T23:15:39.400279+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:55:13.567708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4788224,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":6953,"num_read_kb":14456,"num_write":112,"num_write_kb":530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":102485790,"num_objects":186,"num_object_clones":24,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":15,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":25,"num_read":56627,"num_read_kb":168837,"num_write":7449,"num_write_kb":27494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":564386,"num_keys_recovered":25,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13467648,"data_stored":12951022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6199,"ondisk_log_size":6199,"up":49,"acting":53,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":10,"acting":15,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":1,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37116,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334724,"statfs":{"total":96636764160,"available":96598757376,"internally_reserved":0,"allocated":1810432,"data_stored":1337440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41548,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330292,"statfs":{"total":96636764160,"available":96594219008,"internally_reserved":0,"allocated":2093056,"data_stored":1626179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607567,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607567,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33804,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338036,"statfs":{"total":96636764160,"available":96602148864,"internally_reserved":0,"allocated":2678784,"data_stored":2213232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607568,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":933888,"data_stored":721433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":20029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":319503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2985984,"data_stored":2833382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5107712,"data_stored":4948689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4997120,"data_stored":4829403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:15:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:42.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:42.801 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:42.795+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:42.902 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:42.900+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:42.910 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:42.924 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:42.990 INFO:teuthology.orchestra.run.smithi170.stdout:1978 read 0x6834600 thru 0x683b0f8 (0x6af9 bytes) 2024-09-21T23:15:42.992 INFO:teuthology.orchestra.run.smithi170.stdout:1979 punch from 0x7b6a000 to 0x7b75200, (0xb200 bytes) 2024-09-21T23:15:43.003 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:43.000+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:43.059 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:43.059 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:43.073 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:43.085 INFO:teuthology.orchestra.run.smithi170.stdout:1980 trunc from 0xea7ce00 to 0x471200 2024-09-21T23:15:43.104 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:43.100+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:43.180 INFO:teuthology.orchestra.run.smithi170.stdout:1981 write 0xde18c00 thru 0xde1a702 (0x1b03 bytes) 2024-09-21T23:15:43.182 INFO:teuthology.orchestra.run.smithi170.stdout:1983 write 0xb7ebe00 thru 0xb7eff1f (0x4120 bytes) 2024-09-21T23:15:43.182 INFO:teuthology.orchestra.run.smithi170.stdout:1985 write 0xa32c600 thru 0xa332dfc (0x67fd bytes) 2024-09-21T23:15:43.182 INFO:teuthology.orchestra.run.smithi170.stdout:1986 read 0x816ba00 thru 0x816e42c (0x2a2d bytes) 2024-09-21T23:15:43.204 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:43.204+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:43.205 INFO:teuthology.orchestra.run.smithi170.stdout:1987 trunc from 0xde1a800 to 0x94c000 2024-09-21T23:15:43.209 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:43.209 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:43.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:43.241 INFO:teuthology.orchestra.run.smithi170.stdout:1994 punch from 0x5e3800 to 0x5e8000, (0x4800 bytes) 2024-09-21T23:15:43.243 INFO:teuthology.orchestra.run.smithi170.stdout:1996 read 0xff600 thru 0x102d63 (0x3764 bytes) 2024-09-21T23:15:43.249 INFO:teuthology.orchestra.run.smithi170.stdout:1997 read 0x5aa600 thru 0x5b030a (0x5d0b bytes) 2024-09-21T23:15:43.258 INFO:teuthology.orchestra.run.smithi170.stdout:1998 trunc from 0x94c000 to 0x586f000 2024-09-21T23:15:43.271 INFO:teuthology.orchestra.run.smithi170.stdout:1999 trunc from 0x586f000 to 0x5296600 2024-09-21T23:15:43.291 INFO:teuthology.orchestra.run.smithi170.stdout:2000 read 0x442b600 thru 0x4434a13 (0x9414 bytes) 2024-09-21T23:15:43.291 INFO:teuthology.orchestra.run.smithi170.stdout:2003 punch from 0xc9b800 to 0xca5400, (0x9c00 bytes) 2024-09-21T23:15:43.305 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:43.299+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:43.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:43.324 INFO:teuthology.orchestra.run.smithi170.stdout:2004 write 0xb234a00 thru 0xb242550 (0xdb51 bytes) 2024-09-21T23:15:43.325 INFO:teuthology.orchestra.run.smithi170.stdout:2005 punch from 0x332d400 to 0x3335a00, (0x8600 bytes) 2024-09-21T23:15:43.326 INFO:teuthology.orchestra.run.smithi170.stdout:2007 punch from 0x1f99400 to 0x1f9ee00, (0x5a00 bytes) 2024-09-21T23:15:43.326 INFO:teuthology.orchestra.run.smithi170.stdout:2008 clone 9 order 25 su 1048576 sc 2 2024-09-21T23:15:43.357 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:43.357 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:43.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:43.407 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:43.404+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:43.463 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:43.464 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:43.480 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:43.480 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:43.500 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:43.500 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:43.507 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:43.504+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:43.521 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:43.521 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:43.539 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:43.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:43.539 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:43.539 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:43.541 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:43.542 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:43.547 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:43.547 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:43.553 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:43.608 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:43.603+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:43.681 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:43.681 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:43.681 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:43.681 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:43.681 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:43.695 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:43.739 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:43.735+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:43.809 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:43.803+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:43.861 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:43.861 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:43.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.9045591210000001, 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:43.862 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:15:43.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:15:43.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.9042921370000006, 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:43.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.873 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3357 3.11 3:8fb7664f:::rbd_data.10f1cfc9677.0000000000000015:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6853382940000001, 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049070639999999999, 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.874 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3357 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173029+0000", 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.494e-06 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173037+0000", 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173060+0000", 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717999999999998e-05 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173074+0000", 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4671000000000001e-05 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204211+0000", 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031136891999999999 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204347+0000", 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013595400000000001 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222098+0000", 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017750717999999999 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3418 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6773195300000001, 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054653000000003, 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.878 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3418 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181047+0000", 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6899999999999996e-07 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181057+0000", 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0223e-05 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181064+0000", 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9949999999999999e-06 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181069+0000", 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4160000000000003e-06 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221944+0000", 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040874684000000001 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222101+0000", 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015656599999999999 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3419 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6772885359999998, 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041067436999999998, 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.882 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3419 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181080+0000", 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.402e-06 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181087+0000", 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181092+0000", 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5720000000000002e-06 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222136+0000", 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041044316999999997 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222144+0000", 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9349999999999994e-06 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3420 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6772661129999999, 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041092287999999998, 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3420 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181100+0000", 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8500000000000001e-07 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181102+0000", 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2879999999999999e-06 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181109+0000", 2024-09-21T23:15:43.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181115+0000", 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7819999999999999e-06 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222181+0000", 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041065989999999997 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222192+0000", 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.076e-05 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3421 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6772436150000001, 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041102946000000001, 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3421 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:43.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181125+0000", 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5390000000000002e-06 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181132+0000", 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6610000000000003e-06 2024-09-21T23:15:43.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181137+0000", 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2429999999999996e-06 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222217+0000", 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041080113000000001 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222225+0000", 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9049999999999997e-06 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.893 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3422 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.677221034, 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041150084000000003, 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3422 2024-09-21T23:15:43.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181145+0000", 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:15:43.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181148+0000", 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.57e-06 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181154+0000", 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9909999999999999e-06 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181160+0000", 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222287+0000", 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041126708999999997 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222295+0000", 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6799999999999993e-06 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3424 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:43.897 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6771981629999999, 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041160906999999997, 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3424 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181168+0000", 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181170+0000", 2024-09-21T23:15:43.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181178+0000", 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181183+0000", 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3410000000000002e-06 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222321+0000", 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041137765 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222328+0000", 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6550000000000004e-06 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3425 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:43.901 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6771753079999998, 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041445086999999999, 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3425 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:43.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181191+0000", 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999999e-07 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181193+0000", 2024-09-21T23:15:43.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6800000000000002e-06 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181200+0000", 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6760000000000001e-06 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181205+0000", 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.383e-06 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222627+0000", 2024-09-21T23:15:43.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041421342 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222635+0000", 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6130000000000004e-06 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3500 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181435+0000", 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6769305540000001, 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040921304999999998, 2024-09-21T23:15:43.905 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3500 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181435+0000", 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181435+0000", 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181436+0000", 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1900000000000002e-07 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181438+0000", 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.249e-06 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181444+0000", 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.212e-06 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181449+0000", 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3029999999999999e-06 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222348+0000", 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040899020000000001 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222356+0000", 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9019999999999999e-06 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3508 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6768187340000003, 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040930986000000003, 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.909 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3508 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.61e-07 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181550+0000", 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4250000000000001e-06 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181559+0000", 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7409999999999998e-06 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181564+0000", 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1889999999999998e-06 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222470+0000", 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040906629 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222478+0000", 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3409999999999997e-06 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3509 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.676794664, 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040950634, 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.913 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3509 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181574+0000", 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.199e-06 2024-09-21T23:15:43.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181580+0000", 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0560000000000001e-06 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181585+0000", 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1669999999999998e-06 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222516+0000", 2024-09-21T23:15:43.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931568000000002 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222521+0000", 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3510 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.676771923, 2024-09-21T23:15:43.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040943715999999998, 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3510 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.32e-07 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181596+0000", 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.339e-06 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181603+0000", 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181608+0000", 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222534+0000", 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040925221999999997 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222537+0000", 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.921 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3511 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181615+0000", 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.676750196, 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040936866000000002, 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3511 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181615+0000", 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181615+0000", 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181616+0000", 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181621+0000", 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7010000000000001e-06 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181644+0000", 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2764000000000001e-05 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181651+0000", 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222548+0000", 2024-09-21T23:15:43.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040897385000000001 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222552+0000", 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.698e-06 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3558 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181864+0000", 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6765012490000002, 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040921684, 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.925 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3558 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181864+0000", 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181864+0000", 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181865+0000", 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181867+0000", 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181873+0000", 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.674e-06 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181880+0000", 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222760+0000", 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040879685999999998 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222786+0000", 2024-09-21T23:15:43.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6159000000000001e-05 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3560 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6764785010000001, 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928550000000001, 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3560 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2100000000000002e-07 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181890+0000", 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3130000000000001e-06 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181897+0000", 2024-09-21T23:15:43.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9650000000000002e-06 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181902+0000", 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1429999999999997e-06 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222809+0000", 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040906678000000002 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222816+0000", 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0299999999999996e-06 2024-09-21T23:15:43.932 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3562 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181908+0000", 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6764571300000002, 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931312999999997, 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3562 2024-09-21T23:15:43.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181908+0000", 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181908+0000", 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181909+0000", 2024-09-21T23:15:43.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0200000000000002e-07 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181912+0000", 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6919999999999998e-06 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181918+0000", 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5540000000000003e-06 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181923+0000", 2024-09-21T23:15:43.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9219999999999996e-06 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222836+0000", 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040912568000000003 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222840+0000", 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9750000000000001e-06 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.936 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3564 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181930+0000", 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6764353189999999, 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040929310000000003, 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3564 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181930+0000", 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181930+0000", 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181931+0000", 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.01e-07 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181937+0000", 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4339999999999997e-06 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181944+0000", 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9120000000000001e-06 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181949+0000", 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4979999999999997e-06 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222852+0000", 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040903025000000003 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222860+0000", 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0400000000000004e-06 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.940 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3567 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 2.6764087889999999, 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040956498000000001, 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3567 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:43.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:43.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7899999999999999e-07 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181959+0000", 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1830000000000001e-06 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181966+0000", 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3119999999999999e-06 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181973+0000", 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222902+0000", 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928961999999999 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222913+0000", 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1552000000000001e-05 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.944 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.945 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.945 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:43.945 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:43.945 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:43.946 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:43.947 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:43.939+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:44.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:15:44.076 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:44.076+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:44.128 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:44.128 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:44.129 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:44.129 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:44.142 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:44.170 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone8 from 0xb242600 (overlap 0x471200) to 0x0 2024-09-21T23:15:44.191 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:44.187+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:44.248 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:44.243+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:44.262 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:44.262 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:44.262 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:44.262 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:44.262 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:44.276 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:44.367 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:44.363+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:44.446 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:44.446 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3198 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 2244608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2567819949999999, 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034480396000000003, 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.447 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3198 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185427+0000", 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8400000000000003e-07 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185430+0000", 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.275e-06 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185439+0000", 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6500000000000002e-06 2024-09-21T23:15:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185454+0000", 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4778e-05 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213252+0000", 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027798665 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213316+0000", 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3701999999999994e-05 2024-09-21T23:15:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219907+0000", 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065904420000000002 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3349 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2179072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2537432509999999, 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029976903999999999, 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3349 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188467+0000", 2024-09-21T23:15:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1390000000000002e-06 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188473+0000", 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0159999999999997e-06 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188478+0000", 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5480000000000003e-06 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218328+0000", 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029849969000000001 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218367+0000", 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9474999999999997e-05 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218442+0000", 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4418999999999993e-05 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3350 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2260992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2537194380000001, 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030043218999999999, 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3350 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188492+0000", 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2419999999999999e-06 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188499+0000", 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0670000000000003e-06 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188504+0000", 2024-09-21T23:15:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9699999999999998e-06 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218457+0000", 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029953832999999999 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218493+0000", 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5216000000000003e-05 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218532+0000", 2024-09-21T23:15:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9267000000000002e-05 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3351 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2342912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2536973630000001, 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030166889999999998, 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3351 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188514+0000", 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5210000000000001e-06 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188520+0000", 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4799999999999998e-06 2024-09-21T23:15:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188526+0000", 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3179999999999997e-06 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218555+0000", 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029059 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218613+0000", 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8084999999999997e-05 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218678+0000", 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5021e-05 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3352 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2424832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2536738440000001, 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.03030147, 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3352 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188535+0000", 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188540+0000", 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6790000000000002e-06 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188547+0000", 2024-09-21T23:15:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2439999999999998e-06 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188551+0000", 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7310000000000001e-06 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218703+0000", 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030151305999999999 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218768+0000", 2024-09-21T23:15:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5512999999999994e-05 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218836+0000", 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7550000000000002e-05 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3353 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2506752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2536475180000002, 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030431269, 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3353 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8599999999999998e-07 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188564+0000", 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4229999999999999e-06 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188570+0000", 2024-09-21T23:15:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1199999999999999e-06 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188577+0000", 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.221e-06 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218860+0000", 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030282878999999999 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218924+0000", 2024-09-21T23:15:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3948999999999994e-05 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218992+0000", 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190999999999998e-05 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3354 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2588672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2536250959999999, 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030565452, 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3354 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188586+0000", 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1919999999999999e-06 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188592+0000", 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2380000000000002e-06 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188597+0000", 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0819999999999998e-06 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219018+0000", 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030420823 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219084+0000", 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6513000000000005e-05 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219149+0000", 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4187000000000001e-05 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3460 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2526581939999999, 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029906078999999999, 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3460 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189553+0000", 2024-09-21T23:15:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3010000000000001e-06 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189559+0000", 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189564+0000", 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1819999999999997e-06 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219323+0000", 2024-09-21T23:15:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029758623000000001 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219391+0000", 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8330000000000005e-05 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219456+0000", 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4747999999999997e-05 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3462 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.252613078, 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030022725, 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3462 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189596+0000", 2024-09-21T23:15:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189598+0000", 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8260000000000001e-06 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189605+0000", 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6919999999999996e-06 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189611+0000", 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2360000000000001e-06 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219480+0000", 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029868714000000001 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219547+0000", 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7268999999999996e-05 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219618+0000", 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0508999999999999e-05 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3469 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.252140448, 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029238999999999, 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3469 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190069+0000", 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2500000000000004e-07 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190071+0000", 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9520000000000001e-06 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190075+0000", 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8420000000000004e-06 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190080+0000", 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4800000000000003e-06 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219961+0000", 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029880803000000001 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220030+0000", 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8856999999999995e-05 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220097+0000", 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7379999999999995e-05 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3470 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2521221890000001, 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030180927999999999, 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3470 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190087+0000", 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5030000000000001e-06 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190090+0000", 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.278e-06 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190095+0000", 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8910000000000003e-06 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190100+0000", 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9710000000000003e-06 2024-09-21T23:15:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220128+0000", 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030028195000000001 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220198+0000", 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0405000000000001e-05 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220267+0000", 2024-09-21T23:15:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8684999999999997e-05 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3472 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.252082111, 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030301255999999999, 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3472 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190127+0000", 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.238e-06 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190130+0000", 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190135+0000", 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0270000000000003e-06 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190140+0000", 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8609999999999997e-06 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220297+0000", 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030157498000000001 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220368+0000", 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0811999999999999e-05 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220427+0000", 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9234999999999998e-05 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3484 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3538944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2520039249999999, 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030347946000000001, 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.498 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3484 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190205+0000", 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.13e-06 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190222+0000", 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6909999999999999e-05 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190229+0000", 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2979999999999997e-06 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190234+0000", 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7130000000000002e-06 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220443+0000", 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030209037000000001 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220490+0000", 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6198000000000001e-05 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220552+0000", 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.266e-05 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.502 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3486 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.251946325, 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030430263999999999, 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3486 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:15:44.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190264+0000", 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.007e-06 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190270+0000", 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190275+0000", 2024-09-21T23:15:44.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220569+0000", 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030294237000000002 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220625+0000", 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5902000000000001e-05 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220692+0000", 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6796000000000002e-05 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3487 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2519261500000001, 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030517598, 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.507 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3487 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190284+0000", 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.66e-06 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190287+0000", 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9359999999999999e-06 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190292+0000", 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5219999999999998e-06 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190297+0000", 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0900000000000004e-06 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220721+0000", 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030423823999999999 2024-09-21T23:15:44.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220759+0000", 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7962e-05 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220800+0000", 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0604e-05 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.511 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3489 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3784704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2518852069999999, 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030596542000000001, 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3489 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190324+0000", 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190327+0000", 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6180000000000002e-06 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190333+0000", 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7259999999999999e-06 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190338+0000", 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1680000000000003e-06 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220825+0000", 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030487104000000001 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220863+0000", 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8563000000000003e-05 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220920+0000", 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6051999999999998e-05 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.515 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3490 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3866624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2518634799999999, 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030740849000000001, 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3490 2024-09-21T23:15:44.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190346+0000", 2024-09-21T23:15:44.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5859999999999999e-06 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190349+0000", 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190354+0000", 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1950000000000002e-06 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190359+0000", 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9819999999999999e-06 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220944+0000", 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030585309000000001 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221006+0000", 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1802000000000002e-05 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221086+0000", 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9314999999999995e-05 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3491 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2518281149999999, 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030871663000000001, 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3491 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190382+0000", 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.522e-06 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190384+0000", 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190391+0000", 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2899999999999999e-06 2024-09-21T23:15:44.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190396+0000", 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9570000000000001e-06 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221112+0000", 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030716753999999999 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221179+0000", 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6122000000000002e-05 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221252+0000", 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3152999999999996e-05 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3493 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.251784432, 2024-09-21T23:15:44.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030983984999999999, 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3493 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2800000000000002e-07 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190427+0000", 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4660000000000002e-06 2024-09-21T23:15:44.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190433+0000", 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3110000000000002e-06 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190438+0000", 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7539999999999997e-06 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221282+0000", 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030844399000000002 2024-09-21T23:15:44.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221345+0000", 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3145999999999995e-05 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221408+0000", 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2481e-05 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3495 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.2517416900000002, 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031095845, 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3495 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.529 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:44.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190468+0000", 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190470+0000", 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5509999999999998e-06 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190475+0000", 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190481+0000", 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4389999999999999e-06 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221436+0000", 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030955580999999999 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221501+0000", 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5030999999999995e-05 2024-09-21T23:15:44.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221562+0000", 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1061000000000004e-05 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:44.533 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:44.535 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:44.531+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:44.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:44.636 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:44.636+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:44.673 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:44.673 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:44.673 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:44.673 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:44.687 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:44.737 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:44.731+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:44.794 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-21T23:15:44.837 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:44.831+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:44.857 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:44.857 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:44.857 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:44.857 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:44.858 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:44.873 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:44.938 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:44.936+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:44.987 INFO:teuthology.orchestra.run.smithi170.stdout:2009 trunc from 0xb242600 to 0xac9b400 2024-09-21T23:15:44.997 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.650835217999997, 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.883192856999997, 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.503882775000001, 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.376665762999998, 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.461902032000001, 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:15:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.006949265999999, 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:15:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:15:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.156922883, 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:15:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:15:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.156881465000001, 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:15:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.974795538999999, 2024-09-21T23:15:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.832853195999999, 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.676986900999999, 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:15:45.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.514256624000001, 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:15:45.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:15:45.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:15:45.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.994831241, 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:45.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:45.068 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.994702765, 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:45.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:15:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.993741061000001, 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.99272985, 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:15:45.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:15:45.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:15:45.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.991647903, 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.084 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:15:45.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:15:45.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.115069645, 2024-09-21T23:15:45.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.114730075000001, 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:15:45.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:596 3.12 3:4d026f6e:::rbd_data.10f1dc9013.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.045015063999999, 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078799430000000004, 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:45.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 596 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951020+0000", 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-06 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951929+0000", 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090880699999999997 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951965+0000", 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5734e-05 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951982+0000", 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6806000000000001e-05 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952135+0000", 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015325899999999999 2024-09-21T23:15:45.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952560+0000", 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042476500000000002 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.953181+0000", 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062147299999999999 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.957005+0000", 2024-09-21T23:15:45.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038238 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958342+0000", 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001336622 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958557+0000", 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021509599999999999 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958897+0000", 2024-09-21T23:15:45.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034014100000000002 2024-09-21T23:15:45.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.103 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:45.104 INFO:teuthology.orchestra.run.smithi170.stdout:2010 trunc from 0xac9b400 to 0xd6a7800 2024-09-21T23:15:45.104 INFO:teuthology.orchestra.run.smithi170.stdout:2011 write 0x4ac5c00 thru 0x4ad2d4a (0xd14b bytes) 2024-09-21T23:15:45.105 INFO:teuthology.orchestra.run.smithi170.stdout:2013 clone 10 order 21 su 1048576 sc 9 2024-09-21T23:15:45.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:45.106 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:15:45.106 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:45.104+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:45.204 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:45.204+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:45.277 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:45.277 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:45.277 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:45.277 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:45.291 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:45.306 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:45.299+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:45.407 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:45.403+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:45.461 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:45.461 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:45.461 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:45.461 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:45.461 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:45.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:45.520 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:45.520+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:45.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:45.606 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.678448239, 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677427392, 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677367718999999, 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:15:45.616 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677277742999999, 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.621 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:15:45.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.625 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677227833, 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.626 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:15:45.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:15:45.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677171693, 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.631 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:15:45.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:15:45.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.67711791, 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.636 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:15:45.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:15:45.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677085634000001, 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:15:45.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677055354, 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:45.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.677025698, 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:45.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:15:45.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:15:45.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:15:45.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676995731, 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:15:45.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676917902, 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:45.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:15:45.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:15:45.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676870194999999, 2024-09-21T23:15:45.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:15:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.671 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676839999, 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676810081999999, 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676780281999999, 2024-09-21T23:15:45.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:15:45.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:15:45.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:15:45.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676750349000001, 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:15:45.686 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:15:45.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676685386000001, 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:15:45.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:15:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:45.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.67665491, 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.676625745999999, 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:15:45.701 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:15:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:15:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:15:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:45.706 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:45.708 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:45.708 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:45.708 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:45.709 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:45.709 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:45.704+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:45.710 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:45.710 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:45.729 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:45.729 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:45.749 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:45.749 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:45.767 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:45.767 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:45.770 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:45.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:45.808 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:45.803+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:45.841 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:45.841 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:45.855 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:45.943 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:45.939+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:46.010 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:46.008+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:46.021 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:46.021 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:46.035 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:46.122 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:46.119+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:46.205 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:46.205 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:46.213 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:46.212+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:46.220 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:46.226 INFO:teuthology.orchestra.run.smithi170.stdout:leaving image image_client.0-clone9 intact 2024-09-21T23:15:46.313 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:46.312+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:46.386 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:46.386 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:46.400 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:46.446 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:46.443+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:46.515 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:46.512+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:46.519 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:46.519 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:46.533 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:46.627 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:46.623+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:46.702 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:46.702 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:46.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:46.718 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:46.716+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:46.818 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:46.811+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:46.883 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:46.883 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:46.897 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:46.939 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:46.935+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:46.950 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-21T23:15:47.019 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:47.015+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:47.039 INFO:teuthology.orchestra.run.smithi170.stdout:2015 punch from 0x3d72600 to 0x3d78400, (0x5e00 bytes) 2024-09-21T23:15:47.058 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:47.058 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:47.060 INFO:teuthology.orchestra.run.smithi170.stdout:2016 read 0x80a1800 thru 0x80abf6b (0xa76c bytes) 2024-09-21T23:15:47.073 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:47.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:15:47.127 INFO:teuthology.orchestra.run.smithi170.stdout:2018 trunc from 0xd6a7800 to 0x72f8600 2024-09-21T23:15:47.159 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:47.155+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:47.167 INFO:teuthology.orchestra.run.smithi170.stdout:2019 write 0x5c6f000 thru 0x5c713db (0x23dc bytes) 2024-09-21T23:15:47.169 INFO:teuthology.orchestra.run.smithi170.stdout:2021 clone 11 order 21 su 1048576 sc 11 2024-09-21T23:15:47.221 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:47.215+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:47.246 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:47.246 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:47.261 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:47.322 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:47.315+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:47.377 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:47.377 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:47.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:47.422 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:47.419+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:47.513 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:47.513 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:47.523 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:47.519+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:47.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:47.624 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:47.624+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:47.655 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:47.655 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:47.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:47.722 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5] out_osds: [3, 1] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-21T23:15:47.722 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:15:47.722 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:15:47.722 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-21T23:15:47.722 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T23:15:47.756 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:47.756+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:47.772 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:47.840 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:47.841 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:47.841 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:47.840+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:47.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:47.924 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:47.927 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:47.923+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:47.943 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:47.943 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:47.960 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:47.960 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:47.981 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:47.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:48.002 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:48.002 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:48.018 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:48.018 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:48.018 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:48.023 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:48.024 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:48.028 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:48.028 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:48.028 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:48.028+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:48.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:48.062 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:15:48.129 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:48.128+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.252413337, 2024-09-21T23:15:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251392490000001, 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:15:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251332817, 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:15:48.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:15:48.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251242841, 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:48.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251192931, 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:15:48.202 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:15:48.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:15:48.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251136791, 2024-09-21T23:15:48.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:15:48.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251083008, 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:15:48.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:15:48.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:15:48.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251050731999999, 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:15:48.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.251020452000001, 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:15:48.223 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:15:48.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250990796, 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:15:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250960829, 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:15:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250883, 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:15:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:15:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250835293, 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:15:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:15:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250805097000001, 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.25077518, 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:15:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.25074538, 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250715446999999, 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250650483999999, 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:15:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:15:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:15:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250620008, 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:15:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.250590844, 2024-09-21T23:15:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:48.287 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:48.289 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:48.289 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:48.283+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:48.297 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone10 from 0x72f8600 (overlap 0x72f8600) to 0x397c200 2024-09-21T23:15:48.389 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:48.383+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:48.432 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:48.432 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:48.432 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:48.432 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:48.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:48.490 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:48.488+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:48.591 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:48.587+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:48.620 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:48.620 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:48.620 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:48.620 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:48.620 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:48.635 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:48.692 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:48.692+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:48.793 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:48.787+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:48.809 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:48.809 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:48.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:48.809 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.587513287, 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.810 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:48.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.816 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.075108004000001, 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.817 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:48.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:48.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.075043100999999, 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.824 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:48.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:48.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:48.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.074896662, 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:48.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:48.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.074827546999998, 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.074215221999999, 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.843 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:48.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:48.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.074010817000001, 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.849 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:48.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:48.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.073968031, 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.855 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:48.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:48.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.861 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.683009743, 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.862 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:48.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.867 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682923232999997, 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:48.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:48.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682846343999998, 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:48.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:48.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:48.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:48.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682641504000003, 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:48.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:48.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:48.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682553222000003, 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:48.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:48.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682422394, 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:48.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682376740999999, 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.902 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682338313999999, 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:48.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.913 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682208852000002, 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.914 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:48.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.919 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682187210000002, 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.920 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:48.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:48.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682167165999999, 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.926 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:48.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:48.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:48.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.682083173999999, 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:48.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:48.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:48.937 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:48.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:48.939 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:48.936+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:49.039 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:49.036+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:49.067 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T23:15:49.115 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:49.116 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:49.116 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:49.116 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:49.130 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:49.140 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:49.140+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:49.185 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-21T23:15:49.241 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:49.235+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:49.254 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:49.254 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:49.254 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:49.254 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:49.255 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:49.268 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:49.342 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:49.340+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:49.435 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:49.435 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:49.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:49.435 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:49.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.435 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.477613591000001, 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:15:49.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:15:49.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:49.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:49.441 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.477346606999999, 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:15:49.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:15:49.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:49.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3357 3.11 3:8fb7664f:::rbd_data.10f1cfc9677.0000000000000015:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2583927639999999, 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049070639999999999, 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.447 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3357 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:49.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173029+0000", 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.494e-06 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173037+0000", 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173060+0000", 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717999999999998e-05 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173074+0000", 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4671000000000001e-05 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204211+0000", 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031136891999999999 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204347+0000", 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013595400000000001 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222098+0000", 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017750717999999999 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.451 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3418 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2503740000000008, 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054653000000003, 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3418 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181047+0000", 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6899999999999996e-07 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181057+0000", 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0223e-05 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181064+0000", 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9949999999999999e-06 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181069+0000", 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4160000000000003e-06 2024-09-21T23:15:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221944+0000", 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040874684000000001 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222101+0000", 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015656599999999999 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3419 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2503430059999996, 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041067436999999998, 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3419 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181080+0000", 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.402e-06 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181087+0000", 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181092+0000", 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5720000000000002e-06 2024-09-21T23:15:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222136+0000", 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041044316999999997 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222144+0000", 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9349999999999994e-06 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3420 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2503205830000006, 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041092287999999998, 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3420 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181100+0000", 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8500000000000001e-07 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181102+0000", 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2879999999999999e-06 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181109+0000", 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181115+0000", 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7819999999999999e-06 2024-09-21T23:15:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222181+0000", 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041065989999999997 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222192+0000", 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.076e-05 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3421 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2502980850000007, 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041102946000000001, 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3421 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:15:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181125+0000", 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5390000000000002e-06 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181132+0000", 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6610000000000003e-06 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181137+0000", 2024-09-21T23:15:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2429999999999996e-06 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222217+0000", 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041080113000000001 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222225+0000", 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9049999999999997e-06 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3422 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2502755039999993, 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041150084000000003, 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3422 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181145+0000", 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:15:49.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181148+0000", 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.57e-06 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181154+0000", 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9909999999999999e-06 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181160+0000", 2024-09-21T23:15:49.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222287+0000", 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041126708999999997 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222295+0000", 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6799999999999993e-06 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3424 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2502526330000006, 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041160906999999997, 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3424 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181168+0000", 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:15:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181170+0000", 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181178+0000", 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181183+0000", 2024-09-21T23:15:49.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3410000000000002e-06 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222321+0000", 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041137765 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222328+0000", 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6550000000000004e-06 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3425 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2502297779999996, 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041445086999999999, 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3425 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181191+0000", 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999999e-07 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181193+0000", 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6800000000000002e-06 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181200+0000", 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6760000000000001e-06 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181205+0000", 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.383e-06 2024-09-21T23:15:49.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222627+0000", 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041421342 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222635+0000", 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6130000000000004e-06 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3500 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181435+0000", 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2499850240000008, 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040921304999999998, 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3500 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181435+0000", 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181435+0000", 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181436+0000", 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1900000000000002e-07 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181438+0000", 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.249e-06 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181444+0000", 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.212e-06 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181449+0000", 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3029999999999999e-06 2024-09-21T23:15:49.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222348+0000", 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040899020000000001 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222356+0000", 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9019999999999999e-06 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3508 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.249873204, 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040930986000000003, 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3508 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181547+0000", 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.61e-07 2024-09-21T23:15:49.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181550+0000", 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4250000000000001e-06 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181559+0000", 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7409999999999998e-06 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181564+0000", 2024-09-21T23:15:49.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1889999999999998e-06 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222470+0000", 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040906629 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222478+0000", 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3409999999999997e-06 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3509 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2498491339999998, 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040950634, 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3509 2024-09-21T23:15:49.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:49.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181574+0000", 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.199e-06 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181580+0000", 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0560000000000001e-06 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181585+0000", 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1669999999999998e-06 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222516+0000", 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931568000000002 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222521+0000", 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.491 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3510 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2498263929999993, 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040943715999999998, 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.492 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3510 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.32e-07 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181596+0000", 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.339e-06 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181603+0000", 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181608+0000", 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222534+0000", 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040925221999999997 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222537+0000", 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.495 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3511 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181615+0000", 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2498046659999993, 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040936866000000002, 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3511 2024-09-21T23:15:49.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181615+0000", 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181615+0000", 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181616+0000", 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181621+0000", 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7010000000000001e-06 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181644+0000", 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2764000000000001e-05 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181651+0000", 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222548+0000", 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040897385000000001 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222552+0000", 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.698e-06 2024-09-21T23:15:49.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3558 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181864+0000", 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.249555719, 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040921684, 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3558 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181864+0000", 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181864+0000", 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181865+0000", 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6300000000000001e-07 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181867+0000", 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181873+0000", 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.674e-06 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181880+0000", 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6170000000000003e-06 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222760+0000", 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040879685999999998 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222786+0000", 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6159000000000001e-05 2024-09-21T23:15:49.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3560 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2495329710000007, 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928550000000001, 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.504 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3560 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181887+0000", 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2100000000000002e-07 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181890+0000", 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3130000000000001e-06 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181897+0000", 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9650000000000002e-06 2024-09-21T23:15:49.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181902+0000", 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1429999999999997e-06 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222809+0000", 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040906678000000002 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222816+0000", 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0299999999999996e-06 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3562 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181908+0000", 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2495115999999999, 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931312999999997, 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3562 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181908+0000", 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181908+0000", 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181909+0000", 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0200000000000002e-07 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181912+0000", 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6919999999999998e-06 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181918+0000", 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5540000000000003e-06 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181923+0000", 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9219999999999996e-06 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222836+0000", 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040912568000000003 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222840+0000", 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9750000000000001e-06 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3564 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181930+0000", 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2494897890000001, 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040929310000000003, 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.512 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3564 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181930+0000", 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181930+0000", 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181931+0000", 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.01e-07 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181937+0000", 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4339999999999997e-06 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181944+0000", 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9120000000000001e-06 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181949+0000", 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4979999999999997e-06 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222852+0000", 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040903025000000003 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222860+0000", 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0400000000000004e-06 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3567 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2494632590000005, 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040956498000000001, 2024-09-21T23:15:49.516 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3567 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7899999999999999e-07 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181959+0000", 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1830000000000001e-06 2024-09-21T23:15:49.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181966+0000", 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3119999999999999e-06 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181973+0000", 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222902+0000", 2024-09-21T23:15:49.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928961999999999 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222913+0000", 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1552000000000001e-05 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:49.520 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:49.522 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:49.522 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:49.515+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:49.622 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:49.620+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:49.702 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:49.702 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:49.702 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:49.702 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:49.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:49.763 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:49.759+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:49.824 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:49.819+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:49.842 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:49.843 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:49.858 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:49.943 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:49.939+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:50.026 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:50.023+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:50.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:50.035 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3198 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 2244608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8461925780000001, 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034480396000000003, 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3198 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185427+0000", 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8400000000000003e-07 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185430+0000", 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.275e-06 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185439+0000", 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6500000000000002e-06 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185454+0000", 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4778e-05 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213252+0000", 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027798665 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213316+0000", 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3701999999999994e-05 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219907+0000", 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065904420000000002 2024-09-21T23:15:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3349 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2179072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8431538340000007, 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029976903999999999, 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3349 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188467+0000", 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1390000000000002e-06 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188473+0000", 2024-09-21T23:15:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0159999999999997e-06 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188478+0000", 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5480000000000003e-06 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218328+0000", 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029849969000000001 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218367+0000", 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9474999999999997e-05 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218442+0000", 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4418999999999993e-05 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3350 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2260992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8431300210000003, 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030043218999999999, 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3350 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188492+0000", 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2419999999999999e-06 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188499+0000", 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0670000000000003e-06 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188504+0000", 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9699999999999998e-06 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218457+0000", 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029953832999999999 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218493+0000", 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5216000000000003e-05 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218532+0000", 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9267000000000002e-05 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3351 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2342912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8431079459999999, 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030166889999999998, 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3351 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188514+0000", 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5210000000000001e-06 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188520+0000", 2024-09-21T23:15:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4799999999999998e-06 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188526+0000", 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3179999999999997e-06 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218555+0000", 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029059 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218613+0000", 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8084999999999997e-05 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218678+0000", 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5021e-05 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3352 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2424832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8430844270000009, 2024-09-21T23:15:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.03030147, 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3352 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188535+0000", 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188540+0000", 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6790000000000002e-06 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188547+0000", 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2439999999999998e-06 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188551+0000", 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7310000000000001e-06 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218703+0000", 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030151305999999999 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218768+0000", 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5512999999999994e-05 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218836+0000", 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7550000000000002e-05 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3353 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2506752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8430581010000004, 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030431269, 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3353 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8599999999999998e-07 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188564+0000", 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4229999999999999e-06 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188570+0000", 2024-09-21T23:15:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1199999999999999e-06 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188577+0000", 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.221e-06 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218860+0000", 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030282878999999999 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218924+0000", 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3948999999999994e-05 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218992+0000", 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190999999999998e-05 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3354 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2588672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8430356789999998, 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030565452, 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3354 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188586+0000", 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1919999999999999e-06 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188592+0000", 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2380000000000002e-06 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188597+0000", 2024-09-21T23:15:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0819999999999998e-06 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219018+0000", 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030420823 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219084+0000", 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6513000000000005e-05 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219149+0000", 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4187000000000001e-05 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3460 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8420687769999997, 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029906078999999999, 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3460 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189553+0000", 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3010000000000001e-06 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189559+0000", 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189564+0000", 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1819999999999997e-06 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219323+0000", 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029758623000000001 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219391+0000", 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8330000000000005e-05 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219456+0000", 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4747999999999997e-05 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3462 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8420236610000007, 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030022725, 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3462 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189596+0000", 2024-09-21T23:15:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189598+0000", 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8260000000000001e-06 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189605+0000", 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6919999999999996e-06 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189611+0000", 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2360000000000001e-06 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219480+0000", 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029868714000000001 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219547+0000", 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7268999999999996e-05 2024-09-21T23:15:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219618+0000", 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0508999999999999e-05 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3469 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8415510309999998, 2024-09-21T23:15:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029238999999999, 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3469 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190069+0000", 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2500000000000004e-07 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190071+0000", 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9520000000000001e-06 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190075+0000", 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8420000000000004e-06 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190080+0000", 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4800000000000003e-06 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219961+0000", 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029880803000000001 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220030+0000", 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8856999999999995e-05 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220097+0000", 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7379999999999995e-05 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3470 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8415327720000008, 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030180927999999999, 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3470 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190087+0000", 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5030000000000001e-06 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190090+0000", 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.278e-06 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190095+0000", 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8910000000000003e-06 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190100+0000", 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9710000000000003e-06 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220128+0000", 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030028195000000001 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220198+0000", 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0405000000000001e-05 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220267+0000", 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8684999999999997e-05 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3472 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8414926939999994, 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030301255999999999, 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3472 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190127+0000", 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.238e-06 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190130+0000", 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190135+0000", 2024-09-21T23:15:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0270000000000003e-06 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190140+0000", 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8609999999999997e-06 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220297+0000", 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030157498000000001 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220368+0000", 2024-09-21T23:15:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0811999999999999e-05 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220427+0000", 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9234999999999998e-05 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3484 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3538944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8414145079999997, 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030347946000000001, 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3484 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190205+0000", 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.13e-06 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190222+0000", 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6909999999999999e-05 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190229+0000", 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2979999999999997e-06 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190234+0000", 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7130000000000002e-06 2024-09-21T23:15:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220443+0000", 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030209037000000001 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220490+0000", 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6198000000000001e-05 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220552+0000", 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.266e-05 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3486 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8413569079999998, 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030430263999999999, 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3486 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190264+0000", 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.007e-06 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190270+0000", 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190275+0000", 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220569+0000", 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030294237000000002 2024-09-21T23:15:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220625+0000", 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5902000000000001e-05 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220692+0000", 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6796000000000002e-05 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3487 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8413367330000003, 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030517598, 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3487 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190284+0000", 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.66e-06 2024-09-21T23:15:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190287+0000", 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9359999999999999e-06 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190292+0000", 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5219999999999998e-06 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190297+0000", 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0900000000000004e-06 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220721+0000", 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030423823999999999 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220759+0000", 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7962e-05 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220800+0000", 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0604e-05 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3489 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3784704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8412957900000002, 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030596542000000001, 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3489 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190324+0000", 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190327+0000", 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6180000000000002e-06 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190333+0000", 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7259999999999999e-06 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190338+0000", 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1680000000000003e-06 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220825+0000", 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030487104000000001 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220863+0000", 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8563000000000003e-05 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220920+0000", 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6051999999999998e-05 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3490 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3866624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8412740630000002, 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030740849000000001, 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3490 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190346+0000", 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5859999999999999e-06 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190349+0000", 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190354+0000", 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1950000000000002e-06 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190359+0000", 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9819999999999999e-06 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220944+0000", 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030585309000000001 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221006+0000", 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1802000000000002e-05 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221086+0000", 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9314999999999995e-05 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.112 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3491 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8412386979999997, 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030871663000000001, 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.113 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3491 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190382+0000", 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.522e-06 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190384+0000", 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190391+0000", 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2899999999999999e-06 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190396+0000", 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9570000000000001e-06 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221112+0000", 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030716753999999999 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221179+0000", 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6122000000000002e-05 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221252+0000", 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3152999999999996e-05 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3493 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8411950150000003, 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030983984999999999, 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.117 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3493 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2800000000000002e-07 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190427+0000", 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4660000000000002e-06 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190433+0000", 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3110000000000002e-06 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190438+0000", 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7539999999999997e-06 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221282+0000", 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030844399000000002 2024-09-21T23:15:50.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221345+0000", 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3145999999999995e-05 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221408+0000", 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2481e-05 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3495 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.8411522730000005, 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031095845, 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3495 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:50.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190468+0000", 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:15:50.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190470+0000", 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5509999999999998e-06 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190475+0000", 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190481+0000", 2024-09-21T23:15:50.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4389999999999999e-06 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221436+0000", 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030955580999999999 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221501+0000", 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5030999999999995e-05 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221562+0000", 2024-09-21T23:15:50.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1061000000000004e-05 2024-09-21T23:15:50.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.126 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:50.126 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:50.126 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:50.126 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:15:50.128 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:50.128 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:50.124+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:50.224 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:50.224 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:50.228 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:50.223+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:50.255 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:50.257 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:50.257 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:50.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:50.280 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:50.316 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:50.316 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:50.330 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:50.323+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:50.338 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:50.338 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:50.341 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:50.341 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:50.400 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:50.400 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:50.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:50.431 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:50.427+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:50.531 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:50.531 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:50.532 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:50.532+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:50.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:50.589 INFO:teuthology.orchestra.run.smithi170.stdout:2025 write 0x6a2f000 thru 0x6a3cb3a (0xdb3b bytes) 2024-09-21T23:15:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:2026 read 0x444bc00 thru 0x4453d43 (0x8144 bytes) 2024-09-21T23:15:50.593 INFO:teuthology.orchestra.run.smithi170.stdout:2029 write 0x3f3e200 thru 0x3f4c726 (0xe527 bytes) 2024-09-21T23:15:50.616 INFO:teuthology.orchestra.run.smithi170.stdout:2031 trunc from 0x72f8600 to 0xb348800 2024-09-21T23:15:50.633 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:50.627+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:50.662 INFO:teuthology.orchestra.run.smithi170.stdout:2032 read 0x26d6800 thru 0x26e3c43 (0xd444 bytes) 2024-09-21T23:15:50.665 INFO:teuthology.orchestra.run.smithi170.stdout:2033 read 0x2ba0400 thru 0x2ba769f (0x72a0 bytes) 2024-09-21T23:15:50.665 INFO:teuthology.orchestra.run.smithi170.stdout:2035 write 0x532c000 thru 0x533b364 (0xf365 bytes) 2024-09-21T23:15:50.666 INFO:teuthology.orchestra.run.smithi170.stdout:2037 punch from 0x7f75c00 to 0x7f83200, (0xd600 bytes) 2024-09-21T23:15:50.668 INFO:teuthology.orchestra.run.smithi170.stdout:2038 trunc from 0xb348800 to 0xaa6b800 2024-09-21T23:15:50.706 INFO:teuthology.orchestra.run.smithi170.stdout:2040 read 0x1e1ca00 thru 0x1e2ab0c (0xe10d bytes) 2024-09-21T23:15:50.709 INFO:teuthology.orchestra.run.smithi170.stdout:2043 write 0x8a0c000 thru 0x8a187dd (0xc7de bytes) 2024-09-21T23:15:50.709 INFO:teuthology.orchestra.run.smithi170.stdout:2044 read 0x8e30c00 thru 0x8e3819b (0x759c bytes) 2024-09-21T23:15:50.709 INFO:teuthology.orchestra.run.smithi170.stdout:2046 read 0x6a84400 thru 0x6a92f96 (0xeb97 bytes) 2024-09-21T23:15:50.710 INFO:teuthology.orchestra.run.smithi170.stdout:2047 punch from 0x1ca2e00 to 0x1ca3800, (0xa00 bytes) 2024-09-21T23:15:50.711 INFO:teuthology.orchestra.run.smithi170.stdout:2048 read 0x97bb400 thru 0x97c9643 (0xe244 bytes) 2024-09-21T23:15:50.712 INFO:teuthology.orchestra.run.smithi170.stdout:2050 punch from 0x6539c00 to 0x6549a00, (0xfe00 bytes) 2024-09-21T23:15:50.719 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:50.719 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:50.727 INFO:teuthology.orchestra.run.smithi170.stdout:2051 trunc from 0xaa6b800 to 0xd637200 2024-09-21T23:15:50.733 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:50.732+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:50.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:50.751 INFO:teuthology.orchestra.run.smithi170.stdout:2054 write 0x129da00 thru 0x129f1a6 (0x17a7 bytes) 2024-09-21T23:15:50.753 INFO:teuthology.orchestra.run.smithi170.stdout:2055 read 0xa1c4800 thru 0xa1cd9d5 (0x91d6 bytes) 2024-09-21T23:15:50.793 INFO:teuthology.orchestra.run.smithi170.stdout:2057 write 0xe7eb200 thru 0xe7f7282 (0xc083 bytes) 2024-09-21T23:15:50.795 INFO:teuthology.orchestra.run.smithi170.stdout:2060 write 0xbd25c00 thru 0xbd32f34 (0xd335 bytes) 2024-09-21T23:15:50.795 INFO:teuthology.orchestra.run.smithi170.stdout:2061 read 0xd8a4600 thru 0xd8b379d (0xf19e bytes) 2024-09-21T23:15:50.795 INFO:teuthology.orchestra.run.smithi170.stdout:2062 write 0x318b600 thru 0x318fe12 (0x4813 bytes) 2024-09-21T23:15:50.798 INFO:teuthology.orchestra.run.smithi170.stdout:2065 read 0x56eb800 thru 0x56ec897 (0x1098 bytes) 2024-09-21T23:15:50.800 INFO:teuthology.orchestra.run.smithi170.stdout:2068 read 0xb8b5200 thru 0xb8c4912 (0xf713 bytes) 2024-09-21T23:15:50.800 INFO:teuthology.orchestra.run.smithi170.stdout:2069 read 0xa6d6400 thru 0xa6dbf4c (0x5b4d bytes) 2024-09-21T23:15:50.800 INFO:teuthology.orchestra.run.smithi170.stdout:2070 read 0x7c4de00 thru 0x7c4ff9c (0x219d bytes) 2024-09-21T23:15:50.800 INFO:teuthology.orchestra.run.smithi170.stdout:2071 read 0x741e200 thru 0x7420e8c (0x2c8d bytes) 2024-09-21T23:15:50.834 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:50.832+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:50.873 INFO:teuthology.orchestra.run.smithi170.stdout:2074 trunc from 0xe7f7400 to 0x236e800 2024-09-21T23:15:50.898 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:50.898 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:50.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:50.935 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:50.931+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:50.942 INFO:teuthology.orchestra.run.smithi170.stdout:2075 read 0x772000 thru 0x772553 (0x554 bytes) 2024-09-21T23:15:50.948 INFO:teuthology.orchestra.run.smithi170.stdout:2076 punch from 0x962600 to 0x96cc00, (0xa600 bytes) 2024-09-21T23:15:50.948 INFO:teuthology.orchestra.run.smithi170.stdout:2077 read 0x189e400 thru 0x18a956f (0xb170 bytes) 2024-09-21T23:15:50.998 INFO:teuthology.orchestra.run.smithi170.stdout:2078 write 0xd498e00 thru 0xd4a1822 (0x8a23 bytes) 2024-09-21T23:15:51.000 INFO:teuthology.orchestra.run.smithi170.stdout:2079 write 0x6e68600 thru 0x6e6bdce (0x37cf bytes) 2024-09-21T23:15:51.000 INFO:teuthology.orchestra.run.smithi170.stdout:2082 punch from 0xb077600 to 0xb080000, (0x8a00 bytes) 2024-09-21T23:15:51.001 INFO:teuthology.orchestra.run.smithi170.stdout:2086 trunc from 0xd4a1a00 to 0xd81b800 2024-09-21T23:15:51.034 INFO:teuthology.orchestra.run.smithi170.stdout:2087 punch from 0xbaa5800 to 0xbab1a00, (0xc200 bytes) 2024-09-21T23:15:51.035 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:51.031+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:51.036 INFO:teuthology.orchestra.run.smithi170.stdout:2088 write 0x54cec00 thru 0x54cec70 (0x71 bytes) 2024-09-21T23:15:51.036 INFO:teuthology.orchestra.run.smithi170.stdout:2089 clone 12 order 21 su 32768 sc 8 2024-09-21T23:15:51.071 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:15:51.084 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:51.084 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:51.102 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:51.146 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:51.143+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:51.236 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:51.236+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:51.244 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:51.244 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:51.258 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:51.304 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone11 from 0xd81b800 (overlap 0x236e800) to 0x38b000 2024-09-21T23:15:51.350 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:51.347+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:51.437 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:51.438 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:51.438 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:51.431+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:51.452 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:51.542 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:51.539+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:51.625 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:51.625 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:51.640 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:51.640 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:51.635+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:51.741 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:51.740+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:51.802 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:51.802 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:51.816 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:51.842 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:51.840+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:51.942 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:51.942 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:51.942 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:51.939+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:51.958 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:52.043 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:52.040+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:52.134 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:52.134 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:52.144 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:52.139+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:52.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:52.181 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-21T23:15:52.245 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:52.244+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:52.319 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:52.319 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:52.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:52.342 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:52.345 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:52.344+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:52.446 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:52.444+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:52.464 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:52.464 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:52.479 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:52.502 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:52.519 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:52.519 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:52.538 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:52.546 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:52.544+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:52.556 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:52.557 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:52.577 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:52.577 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:52.580 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:52.580 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:52.605 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3198 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 2244608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.417436310999999, 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034480396000000003, 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3198 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185427+0000", 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8400000000000003e-07 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185430+0000", 2024-09-21T23:15:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.275e-06 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185439+0000", 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6500000000000002e-06 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185454+0000", 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4778e-05 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213252+0000", 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027798665 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213316+0000", 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3701999999999994e-05 2024-09-21T23:15:52.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219907+0000", 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065904420000000002 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3349 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2179072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:52.611 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.414397567, 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029976903999999999, 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3349 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:52.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188467+0000", 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1390000000000002e-06 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188473+0000", 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0159999999999997e-06 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188478+0000", 2024-09-21T23:15:52.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5480000000000003e-06 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218328+0000", 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029849969000000001 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218367+0000", 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9474999999999997e-05 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218442+0000", 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4418999999999993e-05 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3350 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2260992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.414373754, 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030043218999999999, 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3350 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188492+0000", 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2419999999999999e-06 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188499+0000", 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0670000000000003e-06 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188504+0000", 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9699999999999998e-06 2024-09-21T23:15:52.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218457+0000", 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029953832999999999 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218493+0000", 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5216000000000003e-05 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218532+0000", 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9267000000000002e-05 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3351 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2342912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.414351678999999, 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030166889999999998, 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.621 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3351 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188514+0000", 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5210000000000001e-06 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188520+0000", 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4799999999999998e-06 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188526+0000", 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3179999999999997e-06 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218555+0000", 2024-09-21T23:15:52.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029059 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218613+0000", 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8084999999999997e-05 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218678+0000", 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5021e-05 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3352 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2424832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.41432816, 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.03030147, 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3352 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188535+0000", 2024-09-21T23:15:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188540+0000", 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6790000000000002e-06 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188547+0000", 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2439999999999998e-06 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188551+0000", 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7310000000000001e-06 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218703+0000", 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030151305999999999 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218768+0000", 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5512999999999994e-05 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218836+0000", 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7550000000000002e-05 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3353 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2506752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.414301834, 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030431269, 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3353 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8599999999999998e-07 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188564+0000", 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4229999999999999e-06 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188570+0000", 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1199999999999999e-06 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188577+0000", 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.221e-06 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218860+0000", 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030282878999999999 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218924+0000", 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3948999999999994e-05 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218992+0000", 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190999999999998e-05 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3354 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2588672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.414279412000001, 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030565452, 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3354 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:15:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188586+0000", 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1919999999999999e-06 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188592+0000", 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2380000000000002e-06 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188597+0000", 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0819999999999998e-06 2024-09-21T23:15:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219018+0000", 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030420823 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219084+0000", 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6513000000000005e-05 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219149+0000", 2024-09-21T23:15:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4187000000000001e-05 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3460 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.413312510000001, 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029906078999999999, 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3460 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189553+0000", 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3010000000000001e-06 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189559+0000", 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189564+0000", 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1819999999999997e-06 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219323+0000", 2024-09-21T23:15:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029758623000000001 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219391+0000", 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8330000000000005e-05 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219456+0000", 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4747999999999997e-05 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3462 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.413267394, 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030022725, 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3462 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189596+0000", 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189598+0000", 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8260000000000001e-06 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189605+0000", 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6919999999999996e-06 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189611+0000", 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2360000000000001e-06 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219480+0000", 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029868714000000001 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219547+0000", 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7268999999999996e-05 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219618+0000", 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0508999999999999e-05 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3469 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412794763999999, 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029238999999999, 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3469 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190069+0000", 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2500000000000004e-07 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190071+0000", 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9520000000000001e-06 2024-09-21T23:15:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190075+0000", 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8420000000000004e-06 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190080+0000", 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4800000000000003e-06 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219961+0000", 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029880803000000001 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220030+0000", 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8856999999999995e-05 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220097+0000", 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7379999999999995e-05 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3470 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412776505, 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030180927999999999, 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3470 2024-09-21T23:15:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190087+0000", 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5030000000000001e-06 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190090+0000", 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.278e-06 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190095+0000", 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8910000000000003e-06 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190100+0000", 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9710000000000003e-06 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220128+0000", 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030028195000000001 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220198+0000", 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0405000000000001e-05 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220267+0000", 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8684999999999997e-05 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3472 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412736427, 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030301255999999999, 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3472 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190127+0000", 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.238e-06 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190130+0000", 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190135+0000", 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0270000000000003e-06 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190140+0000", 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8609999999999997e-06 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220297+0000", 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030157498000000001 2024-09-21T23:15:52.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220368+0000", 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0811999999999999e-05 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220427+0000", 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9234999999999998e-05 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3484 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3538944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412658241000001, 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030347946000000001, 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3484 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.662 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190205+0000", 2024-09-21T23:15:52.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.13e-06 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190222+0000", 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6909999999999999e-05 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190229+0000", 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2979999999999997e-06 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190234+0000", 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7130000000000002e-06 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220443+0000", 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030209037000000001 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220490+0000", 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6198000000000001e-05 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220552+0000", 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.266e-05 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3486 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412600640999999, 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030430263999999999, 2024-09-21T23:15:52.666 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3486 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190264+0000", 2024-09-21T23:15:52.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.007e-06 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190270+0000", 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190275+0000", 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220569+0000", 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030294237000000002 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220625+0000", 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5902000000000001e-05 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220692+0000", 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6796000000000002e-05 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.670 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3487 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412580466, 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030517598, 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3487 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190284+0000", 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.66e-06 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190287+0000", 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9359999999999999e-06 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190292+0000", 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5219999999999998e-06 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190297+0000", 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0900000000000004e-06 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220721+0000", 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030423823999999999 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220759+0000", 2024-09-21T23:15:52.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7962e-05 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220800+0000", 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0604e-05 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3489 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3784704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412539523, 2024-09-21T23:15:52.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030596542000000001, 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3489 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190324+0000", 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190327+0000", 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6180000000000002e-06 2024-09-21T23:15:52.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190333+0000", 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7259999999999999e-06 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190338+0000", 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1680000000000003e-06 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220825+0000", 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030487104000000001 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220863+0000", 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8563000000000003e-05 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220920+0000", 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6051999999999998e-05 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.679 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3490 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3866624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412517795999999, 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030740849000000001, 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3490 2024-09-21T23:15:52.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190346+0000", 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5859999999999999e-06 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190349+0000", 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190354+0000", 2024-09-21T23:15:52.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1950000000000002e-06 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190359+0000", 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9819999999999999e-06 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220944+0000", 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030585309000000001 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221006+0000", 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1802000000000002e-05 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221086+0000", 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9314999999999995e-05 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3491 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:52.684 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412482431000001, 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030871663000000001, 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3491 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190382+0000", 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.522e-06 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190384+0000", 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190391+0000", 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2899999999999999e-06 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190396+0000", 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9570000000000001e-06 2024-09-21T23:15:52.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221112+0000", 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030716753999999999 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221179+0000", 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6122000000000002e-05 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221252+0000", 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3152999999999996e-05 2024-09-21T23:15:52.688 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3493 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412438748, 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030983984999999999, 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3493 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2800000000000002e-07 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190427+0000", 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4660000000000002e-06 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190433+0000", 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3110000000000002e-06 2024-09-21T23:15:52.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190438+0000", 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7539999999999997e-06 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221282+0000", 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030844399000000002 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221345+0000", 2024-09-21T23:15:52.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3145999999999995e-05 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221408+0000", 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2481e-05 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3495 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:52.693 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.412396006, 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031095845, 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3495 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:52.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190468+0000", 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:52.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190470+0000", 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5509999999999998e-06 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190475+0000", 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190481+0000", 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4389999999999999e-06 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:52.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221436+0000", 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030955580999999999 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221501+0000", 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5030999999999995e-05 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221562+0000", 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1061000000000004e-05 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:52.698 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:52.698 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:52.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:52.700 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:52.696+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:52.756 INFO:teuthology.orchestra.run.smithi170.stdout:2090 trunc from 0xd81b800 to 0x1f74000 2024-09-21T23:15:52.799 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:52.795+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:52.851 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:52.851 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:52.851 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:52.851 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:52.851 INFO:teuthology.orchestra.run.smithi170.stdout:2092 write 0x11a9200 thru 0x11af2ec (0x60ed bytes) 2024-09-21T23:15:52.854 INFO:teuthology.orchestra.run.smithi170.stdout:2094 read 0x12ba000 thru 0x12c8777 (0xe778 bytes) 2024-09-21T23:15:52.855 INFO:teuthology.orchestra.run.smithi170.stdout:2096 punch from 0x385600 to 0x388a00, (0x3400 bytes) 2024-09-21T23:15:52.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:52.900 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:52.895+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:52.901 INFO:teuthology.orchestra.run.smithi170.stdout:2097 write 0x3790e00 thru 0x3791913 (0xb14 bytes) 2024-09-21T23:15:52.952 INFO:teuthology.orchestra.run.smithi170.stdout:2098 write 0xa79a200 thru 0xa7a3b29 (0x992a bytes) 2024-09-21T23:15:52.956 INFO:teuthology.orchestra.run.smithi170.stdout:2101 punch from 0x898c00 to 0x89e000, (0x5400 bytes) 2024-09-21T23:15:52.998 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:52.998 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:52.998 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:52.998 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:52.998 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:52.999 INFO:teuthology.orchestra.run.smithi170.stdout:2103 write 0xcc70200 thru 0xcc7dabf (0xd8c0 bytes) 2024-09-21T23:15:53.001 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:52.995+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:53.002 INFO:teuthology.orchestra.run.smithi170.stdout:2104 read 0xcbb5400 thru 0xcbc1bf7 (0xc7f8 bytes) 2024-09-21T23:15:53.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:53.038 INFO:teuthology.orchestra.run.smithi170.stdout:2106 trunc from 0xcc7dc00 to 0x151a800 2024-09-21T23:15:53.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:15:53.117 INFO:teuthology.orchestra.run.smithi170.stdout:2108 write 0x7168200 thru 0x71758ed (0xd6ee bytes) 2024-09-21T23:15:53.119 INFO:teuthology.orchestra.run.smithi170.stdout:2110 punch from 0x64cac00 to 0x64cb600, (0xa00 bytes) 2024-09-21T23:15:53.120 INFO:teuthology.orchestra.run.smithi170.stdout:2111 read 0x835600 thru 0x842957 (0xd358 bytes) 2024-09-21T23:15:53.124 INFO:teuthology.orchestra.run.smithi170.stdout:2112 trunc from 0x7175a00 to 0x594d200 2024-09-21T23:15:53.140 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:53.140+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:53.153 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:53.153 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:53.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.806978458000003, 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:53.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:53.160 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.039336097000003, 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:53.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:53.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.660026015, 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.532809002999997, 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:53.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:53.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.618045272000003, 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:53.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:15:53.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:15:53.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.163092505999998, 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:15:53.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:15:53.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:53.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.313066122999999, 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.196 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.313024704999997, 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:53.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:15:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:15:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.130938778999997, 2024-09-21T23:15:53.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:15:53.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:15:53.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:15:53.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.988996436000001, 2024-09-21T23:15:53.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:15:53.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.833130141000002, 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:15:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.670399864, 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:15:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:15:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.150974480999999, 2024-09-21T23:15:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:15:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:15:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:15:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.150846004999998, 2024-09-21T23:15:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:15:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.149884301, 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:15:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:15:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.148873089999999, 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:15:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:15:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.147791142999999, 2024-09-21T23:15:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:15:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.271212885000001, 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:15:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.270873314999999, 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:15:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:15:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:596 3.12 3:4d026f6e:::rbd_data.10f1dc9013.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.201158304, 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078799430000000004, 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 596 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951020+0000", 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-06 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951929+0000", 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090880699999999997 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951965+0000", 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5734e-05 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951982+0000", 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6806000000000001e-05 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952135+0000", 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015325899999999999 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952560+0000", 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042476500000000002 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.953181+0000", 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062147299999999999 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.957005+0000", 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038238 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958342+0000", 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001336622 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958557+0000", 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021509599999999999 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958897+0000", 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034014100000000002 2024-09-21T23:15:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.271 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:53.273 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:53.273 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:53.272+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2114 read 0x4382c00 thru 0x4388dfd (0x61fe bytes) 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2117 trunc from 0x594d200 to 0x8225c00 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2118 read 0x414ca00 thru 0x414f647 (0x2c48 bytes) 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2119 punch from 0xa86800 to 0xa91000, (0xa800 bytes) 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2121 write 0x9c8000 thru 0x9ccd32 (0x4d33 bytes) 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2122 read 0x6155800 thru 0x6157e67 (0x2668 bytes) 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2126 read 0x4cc8400 thru 0x4ccd57e (0x517f bytes) 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2127 punch from 0x308da00 to 0x308f800, (0x1e00 bytes) 2024-09-21T23:15:53.274 INFO:teuthology.orchestra.run.smithi170.stdout:2128 trunc from 0x8225c00 to 0x3465600 2024-09-21T23:15:53.275 INFO:teuthology.orchestra.run.smithi170.stdout:2129 read 0x177ae00 thru 0x178129c (0x649d bytes) 2024-09-21T23:15:53.275 INFO:teuthology.orchestra.run.smithi170.stdout:2130 trunc from 0x3465600 to 0x7fb3a00 2024-09-21T23:15:53.286 INFO:teuthology.orchestra.run.smithi170.stdout:2131 write 0x136ca00 thru 0x136e3cc (0x19cd bytes) 2024-09-21T23:15:53.291 INFO:teuthology.orchestra.run.smithi170.stdout:2132 write 0x4dfba00 thru 0x4e034c4 (0x7ac5 bytes) 2024-09-21T23:15:53.333 INFO:teuthology.orchestra.run.smithi170.stdout:2133 write 0x8572e00 thru 0x857536f (0x2570 bytes) 2024-09-21T23:15:53.335 INFO:teuthology.orchestra.run.smithi170.stdout:2135 read 0x32ad400 thru 0x32b2801 (0x5402 bytes) 2024-09-21T23:15:53.362 INFO:teuthology.orchestra.run.smithi170.stdout:2136 write 0x8be5c00 thru 0x8be6d3b (0x113c bytes) 2024-09-21T23:15:53.364 INFO:teuthology.orchestra.run.smithi170.stdout:2137 punch from 0x48fd000 to 0x48fea00, (0x1a00 bytes) 2024-09-21T23:15:53.364 INFO:teuthology.orchestra.run.smithi170.stdout:2138 write 0x5cb0600 thru 0x5cbdf54 (0xd955 bytes) 2024-09-21T23:15:53.373 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:53.367+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:53.380 INFO:teuthology.orchestra.run.smithi170.stdout:2139 trunc from 0x8be6e00 to 0xe664200 2024-09-21T23:15:53.446 INFO:teuthology.orchestra.run.smithi170.stdout:2141 write 0xee02600 thru 0xee10743 (0xe144 bytes) 2024-09-21T23:15:53.447 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:53.447 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:53.447 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:53.447 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:53.449 INFO:teuthology.orchestra.run.smithi170.stdout:2145 punch from 0x927e200 to 0x9288000, (0x9e00 bytes) 2024-09-21T23:15:53.450 INFO:teuthology.orchestra.run.smithi170.stdout:2146 read 0x569ce00 thru 0x56a0f68 (0x4169 bytes) 2024-09-21T23:15:53.451 INFO:teuthology.orchestra.run.smithi170.stdout:2147 write 0x4838200 thru 0x4842a22 (0xa823 bytes) 2024-09-21T23:15:53.458 INFO:teuthology.orchestra.run.smithi170.stdout:2151 read 0x197ba00 thru 0x198ad55 (0xf356 bytes) 2024-09-21T23:15:53.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:53.481 INFO:teuthology.orchestra.run.smithi170.stdout:2152 trunc from 0xee10800 to 0x4442a00 2024-09-21T23:15:53.508 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:53.508+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:53.519 INFO:teuthology.orchestra.run.smithi170.stdout:skipping zero length punch hole 2024-09-21T23:15:53.533 INFO:teuthology.orchestra.run.smithi170.stdout:2154 trunc from 0x4442a00 to 0xa8b7a00 2024-09-21T23:15:53.561 INFO:teuthology.orchestra.run.smithi170.stdout:2155 write 0x5f98e00 thru 0x5f9dfd2 (0x51d3 bytes) 2024-09-21T23:15:53.565 INFO:teuthology.orchestra.run.smithi170.stdout:2159 punch from 0x7e1fc00 to 0x7e24e00, (0x5200 bytes) 2024-09-21T23:15:53.565 INFO:teuthology.orchestra.run.smithi170.stdout:2160 write 0x4183400 thru 0x41931ad (0xfdae bytes) 2024-09-21T23:15:53.567 INFO:teuthology.orchestra.run.smithi170.stdout:2161 read 0x2ccba00 thru 0x2cd602c (0xa62d bytes) 2024-09-21T23:15:53.574 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:53.572+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:53.588 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:53.589 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:53.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:53.606 INFO:teuthology.orchestra.run.smithi170.stdout:2162 write 0xc5aca00 thru 0xc5b1333 (0x4934 bytes) 2024-09-21T23:15:53.607 INFO:teuthology.orchestra.run.smithi170.stdout:2163 read 0x30c9200 thru 0x30d51d3 (0xbfd4 bytes) 2024-09-21T23:15:53.618 INFO:teuthology.orchestra.run.smithi170.stdout:2164 trunc from 0xc5b1400 to 0x8c05c00 2024-09-21T23:15:53.658 INFO:teuthology.orchestra.run.smithi170.stdout:2166 write 0x47fa200 thru 0x47fd24b (0x304c bytes) 2024-09-21T23:15:53.675 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:53.671+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:53.676 INFO:teuthology.orchestra.run.smithi170.stdout:2168 trunc from 0x8c05c00 to 0x27c7000 2024-09-21T23:15:53.715 INFO:teuthology.orchestra.run.smithi170.stdout:2169 trunc from 0x27c7000 to 0x2d5e800 2024-09-21T23:15:53.776 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:53.772+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:53.780 INFO:teuthology.orchestra.run.smithi170.stdout:2170 write 0x7f45a00 thru 0x7f4733d (0x193e bytes) 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.852292356, 2024-09-21T23:15:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:15:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.851271509, 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:15:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.851211836000001, 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:15:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.851121859999999, 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:15:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.851071950000001, 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:15:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.85101581, 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850962027000001, 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:15:53.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850929750999999, 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:15:53.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:15:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850899470999998, 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850869814999999, 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:15:53.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:15:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850839848, 2024-09-21T23:15:53.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:15:53.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:15:53.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:53.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850762019000001, 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:15:53.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:15:53.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:53.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850714312000001, 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:15:53.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:15:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850684116, 2024-09-21T23:15:53.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:15:53.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:15:53.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:15:53.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850654199000001, 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:15:53.852 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850624399000001, 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.857 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850594466, 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.862 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850529503000001, 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:53.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850499027000001, 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:15:53.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.850469863000001, 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:53.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:53.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:53.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:53.882 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:53.884 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2171 punch from 0x530f200 to 0x5313600, (0x4400 bytes) 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2172 read 0x24b7a00 thru 0x24bfe7c (0x847d bytes) 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2174 trunc from 0x7f47400 to 0xc079400 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2175 read 0xa4b0600 thru 0xa4b7808 (0x7209 bytes) 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2177 write 0x6887400 thru 0x688a203 (0x2e04 bytes) 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2179 write 0x432b800 thru 0x4335647 (0x9e48 bytes) 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2184 trunc from 0xc079400 to 0x659b200 2024-09-21T23:15:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:2186 trunc from 0x659b200 to 0x90ee200 2024-09-21T23:15:53.886 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:53.879+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:53.919 INFO:teuthology.orchestra.run.smithi170.stdout:2188 trunc from 0x90ee200 to 0x2873800 2024-09-21T23:15:53.947 INFO:teuthology.orchestra.run.smithi170.stdout:2189 punch from 0x2303c00 to 0x2308a00, (0x4e00 bytes) 2024-09-21T23:15:53.947 INFO:teuthology.orchestra.run.smithi170.stdout:2193 read 0x13d2e00 thru 0x13d80c8 (0x52c9 bytes) 2024-09-21T23:15:53.949 INFO:teuthology.orchestra.run.smithi170.stdout:2194 punch from 0x2210600 to 0x221fe00, (0xf800 bytes) 2024-09-21T23:15:53.949 INFO:teuthology.orchestra.run.smithi170.stdout:2196 read 0x10bcc00 thru 0x10c5d56 (0x9157 bytes) 2024-09-21T23:15:53.950 INFO:teuthology.orchestra.run.smithi170.stdout:2197 read 0x1a6e400 thru 0x1a7cccd (0xe8ce bytes) 2024-09-21T23:15:53.952 INFO:teuthology.orchestra.run.smithi170.stdout:2200 read 0x243da00 thru 0x2447e94 (0xa495 bytes) 2024-09-21T23:15:53.976 INFO:teuthology.orchestra.run.smithi170.stdout:2201 trunc from 0x2873800 to 0xb8eca00 2024-09-21T23:15:53.984 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:53.979+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:53.999 INFO:teuthology.orchestra.run.smithi170.stdout:2202 punch from 0x5c4ea00 to 0x5c53a00, (0x5000 bytes) 2024-09-21T23:15:54.000 INFO:teuthology.orchestra.run.smithi170.stdout:2208 punch from 0x1548a00 to 0x1548e00, (0x400 bytes) 2024-09-21T23:15:54.001 INFO:teuthology.orchestra.run.smithi170.stdout:2209 write 0x50eec00 thru 0x50f4798 (0x5b99 bytes) 2024-09-21T23:15:54.001 INFO:teuthology.orchestra.run.smithi170.stdout:2213 read 0xa2f6600 thru 0xa2fe68e (0x808f bytes) 2024-09-21T23:15:54.001 INFO:teuthology.orchestra.run.smithi170.stdout:2214 write 0x6e97200 thru 0x6ea1ebb (0xacbc bytes) 2024-09-21T23:15:54.004 INFO:teuthology.orchestra.run.smithi170.stdout:2215 trunc from 0xb8eca00 to 0xa871a00 2024-09-21T23:15:54.029 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:54.030 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:54.043 INFO:teuthology.orchestra.run.smithi170.stdout:2216 read 0x9e0000 thru 0x9eef00 (0xef01 bytes) 2024-09-21T23:15:54.045 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:54.070 INFO:teuthology.orchestra.run.smithi170.stdout:2217 write 0xba13e00 thru 0xba22e1e (0xf01f bytes) 2024-09-21T23:15:54.071 INFO:teuthology.orchestra.run.smithi170.stdout:2218 write 0x1a7e400 thru 0x1a858be (0x74bf bytes) 2024-09-21T23:15:54.074 INFO:teuthology.orchestra.run.smithi170.stdout:2220 write 0xab40000 thru 0xab4c624 (0xc625 bytes) 2024-09-21T23:15:54.074 INFO:teuthology.orchestra.run.smithi170.stdout:2221 read 0x69e0200 thru 0x69edf34 (0xdd35 bytes) 2024-09-21T23:15:54.075 INFO:teuthology.orchestra.run.smithi170.stdout:2222 read 0xab9a200 thru 0xaba5f73 (0xbd74 bytes) 2024-09-21T23:15:54.087 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:54.083+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:54.089 INFO:teuthology.orchestra.run.smithi170.stdout:2223 trunc from 0xba23000 to 0x5185400 2024-09-21T23:15:54.089 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:15:54.097 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5] out_osds: [3, 1] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-21T23:15:54.097 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-21T23:15:54.097 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T23:15:54.097 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T23:15:54.099 INFO:tasks.ceph.osd.0:Started 2024-09-21T23:15:54.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:15:54.129 INFO:teuthology.orchestra.run.smithi170.stdout:2225 punch from 0xeb0200 to 0xeb0600, (0x400 bytes) 2024-09-21T23:15:54.133 INFO:teuthology.orchestra.run.smithi170.stdout:2226 read 0x384be00 thru 0x384ffee (0x41ef bytes) 2024-09-21T23:15:54.134 INFO:teuthology.orchestra.run.smithi170.stdout:2229 write 0x3d57200 thru 0x3d60bda (0x99db bytes) 2024-09-21T23:15:54.135 INFO:teuthology.orchestra.run.smithi170.stdout:2233 read 0x39af400 thru 0x39b74cc (0x80cd bytes) 2024-09-21T23:15:54.135 INFO:teuthology.orchestra.run.smithi170.stdout:2236 punch from 0x2cb6e00 to 0x2cc3c00, (0xce00 bytes) 2024-09-21T23:15:54.135 INFO:teuthology.orchestra.run.smithi170.stdout:2237 write 0x2627600 thru 0x2628326 (0xd27 bytes) 2024-09-21T23:15:54.170 INFO:teuthology.orchestra.run.smithi170.stdout:2238 write 0xbf8ec00 thru 0xbf9691f (0x7d20 bytes) 2024-09-21T23:15:54.171 INFO:teuthology.orchestra.run.smithi170.stdout:2239 read 0xb86b800 thru 0xb877ba2 (0xc3a3 bytes) 2024-09-21T23:15:54.172 INFO:teuthology.orchestra.run.smithi170.stdout:2242 read 0x3526a00 thru 0x3527977 (0xf78 bytes) 2024-09-21T23:15:54.172 INFO:teuthology.orchestra.run.smithi170.stdout:2244 read 0xb9c1800 thru 0xb9cd1e4 (0xb9e5 bytes) 2024-09-21T23:15:54.185 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:54.184+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:54.188 INFO:teuthology.orchestra.run.smithi170.stdout:2246 trunc from 0xbf96a00 to 0x5fd8a00 2024-09-21T23:15:54.216 INFO:teuthology.orchestra.run.smithi170.stdout:2247 read 0x190a00 thru 0x19868b (0x7c8c bytes) 2024-09-21T23:15:54.217 INFO:teuthology.orchestra.run.smithi170.stdout:2249 punch from 0x5b34200 to 0x5b35000, (0xe00 bytes) 2024-09-21T23:15:54.218 INFO:teuthology.orchestra.run.smithi170.stdout:2251 write 0x1bba000 thru 0x1bbee98 (0x4e99 bytes) 2024-09-21T23:15:54.218 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:54.219 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:15:54.221 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:config diff 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:config diff get 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:config get 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:config help [] 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:config set ... 2024-09-21T23:15:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:admin_socket: invalid command 2024-09-21T23:15:54.224 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:15:54.224 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-21T23:15:54.234 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:54.244 INFO:teuthology.orchestra.run.smithi170.stdout:2253 write 0x8a19800 thru 0x8a215e7 (0x7de8 bytes) 2024-09-21T23:15:54.284 INFO:teuthology.orchestra.run.smithi170.stdout:2254 write 0xdbe4800 thru 0xdbe5cae (0x14af bytes) 2024-09-21T23:15:54.286 INFO:teuthology.orchestra.run.smithi170.stdout:2259 punch from 0x94e1600 to 0x94eb200, (0x9c00 bytes) 2024-09-21T23:15:54.286 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:54.283+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:54.286 INFO:teuthology.orchestra.run.smithi170.stdout:2260 write 0x653f000 thru 0x6545fcd (0x6fce bytes) 2024-09-21T23:15:54.293 INFO:teuthology.orchestra.run.smithi170.stdout:2262 trunc from 0xdbe5e00 to 0xb810200 2024-09-21T23:15:54.323 INFO:teuthology.orchestra.run.smithi170.stdout:2263 write 0x60ae200 thru 0x60bc7ab (0xe5ac bytes) 2024-09-21T23:15:54.324 INFO:teuthology.orchestra.run.smithi170.stdout:2264 read 0x243b600 thru 0x243dd90 (0x2791 bytes) 2024-09-21T23:15:54.347 INFO:teuthology.orchestra.run.smithi170.stdout:2267 trunc from 0xb810200 to 0x298f000 2024-09-21T23:15:54.361 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:54.361 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:54.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:54.361 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 68.138577460999997, 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.362 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:54.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.626172178000004, 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.369 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:54.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.374 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.626107274999995, 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:54.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.380 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.625960836000004, 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:54.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:54.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.386 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.625891721000002, 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:54.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:54.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:54.392 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.625279395999996, 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:54.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:54.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.625074991000005, 2024-09-21T23:15:54.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:54.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:54.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.625032204999997, 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:54.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:54.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.234073917000003, 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:54.410 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:54.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:54.416 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233987407000001, 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:54.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:54.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:54.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233910518000002, 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:54.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:54.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233705678000007, 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.429 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:54.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233617396, 2024-09-21T23:15:54.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:54.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:54.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233486568000004, 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:54.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:54.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233440915000003, 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:54.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233402487999996, 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.454 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233273026000006, 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.460 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:54.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:54.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.466 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233251383999999, 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:54.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:54.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233231340000003, 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.473 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:54.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:54.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.233147348000003, 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.479 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:54.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:54.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.485 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:54.487 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:54.488 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:54.483+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:54.488 INFO:teuthology.orchestra.run.smithi170.stdout:2270 write 0x6beb200 thru 0x6bf0ca8 (0x5aa9 bytes) 2024-09-21T23:15:54.488 INFO:teuthology.orchestra.run.smithi170.stdout:2272 punch from 0x3e2f200 to 0x3e39c00, (0xaa00 bytes) 2024-09-21T23:15:54.488 INFO:teuthology.orchestra.run.smithi170.stdout:2276 read 0x29e5e00 thru 0x29f431a (0xe51b bytes) 2024-09-21T23:15:54.488 INFO:teuthology.orchestra.run.smithi170.stdout:2277 read 0x4b55600 thru 0x4b56139 (0xb3a bytes) 2024-09-21T23:15:54.488 INFO:teuthology.orchestra.run.smithi170.stdout:2278 trunc from 0x6bf0e00 to 0x75ac800 2024-09-21T23:15:54.488 INFO:teuthology.orchestra.run.smithi170.stdout:2279 punch from 0x24fe00 to 0x25ae00, (0xb000 bytes) 2024-09-21T23:15:54.488 INFO:teuthology.orchestra.run.smithi170.stdout:2280 write 0x35f0600 thru 0x35f790c (0x730d bytes) 2024-09-21T23:15:54.503 INFO:teuthology.orchestra.run.smithi170.stdout:2282 trunc from 0x75ac800 to 0x19bf200 2024-09-21T23:15:54.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:54.588 INFO:teuthology.orchestra.run.smithi170.stdout:2283 write 0x6cd0a00 thru 0x6cdf117 (0xe718 bytes) 2024-09-21T23:15:54.600 INFO:teuthology.orchestra.run.smithi170.stdout:2285 trunc from 0x6cdf200 to 0xab4b600 2024-09-21T23:15:54.623 INFO:teuthology.orchestra.run.smithi170.stdout:2287 write 0x5257800 thru 0x525f58a (0x7d8b bytes) 2024-09-21T23:15:54.625 INFO:teuthology.orchestra.run.smithi170.stdout:2288 punch from 0x1722600 to 0x1726800, (0x4200 bytes) 2024-09-21T23:15:54.625 INFO:teuthology.orchestra.run.smithi170.stdout:2289 write 0x4463200 thru 0x446d82d (0xa62e bytes) 2024-09-21T23:15:54.626 INFO:teuthology.orchestra.run.smithi170.stdout:2290 read 0x77c6e00 thru 0x77cc020 (0x5221 bytes) 2024-09-21T23:15:54.626 INFO:teuthology.orchestra.run.smithi170.stdout:2291 punch from 0x67e0e00 to 0x67eee00, (0xe000 bytes) 2024-09-21T23:15:54.628 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:54.628+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:54.652 INFO:teuthology.orchestra.run.smithi170.stdout:2292 write 0xbd71600 thru 0xbd74e24 (0x3825 bytes) 2024-09-21T23:15:54.653 INFO:teuthology.orchestra.run.smithi170.stdout:2293 read 0x2dbec00 thru 0x2dbfa0b (0xe0c bytes) 2024-09-21T23:15:54.653 INFO:teuthology.orchestra.run.smithi170.stdout:2294 write 0xb3cb200 thru 0xb3da695 (0xf496 bytes) 2024-09-21T23:15:54.653 INFO:teuthology.orchestra.run.smithi170.stdout:2299 punch from 0xd06800 to 0xd0f400, (0x8c00 bytes) 2024-09-21T23:15:54.653 INFO:teuthology.orchestra.run.smithi170.stdout:2300 write 0x27f800 thru 0x28db89 (0xe38a bytes) 2024-09-21T23:15:54.656 INFO:teuthology.orchestra.run.smithi170.stdout:2301 write 0x4415200 thru 0x442340f (0xe210 bytes) 2024-09-21T23:15:54.658 INFO:teuthology.orchestra.run.smithi170.stdout:2302 trunc from 0xbd75000 to 0xb5ae400 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:3357 3.11 3:8b22024a:::rbd_data.10f177c8c3f9.0000000000000000:head [read 20480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:54.681024+0000", 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00059880000000000003, 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00068690299999999999, 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3357 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681024+0000", 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681024+0000", 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681026+0000", 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3170000000000001e-06 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681033+0000", 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4889999999999998e-06 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681059+0000", 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5446999999999999e-05 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681070+0000", 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1902999999999999e-05 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681368+0000", 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029742899999999999 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:54.681600+0000", 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023182199999999999 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:15:54.687 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:54.688 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:54.683+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:54.693 INFO:teuthology.orchestra.run.smithi170.stdout:2303 read 0x37a7c00 thru 0x37afc1a (0x801b bytes) 2024-09-21T23:15:54.698 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:54.716 INFO:teuthology.orchestra.run.smithi170.stdout:2305 trunc from 0xb5ae400 to 0xeeaa00 2024-09-21T23:15:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:54.746 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:54.746 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:54.764 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:54.768 INFO:teuthology.orchestra.run.smithi170.stdout:2307 read 0xe8a000 thru 0xe8daf6 (0x3af7 bytes) 2024-09-21T23:15:54.770 INFO:teuthology.orchestra.run.smithi170.stdout:2308 read 0xa31800 thru 0xa33e56 (0x2657 bytes) 2024-09-21T23:15:54.771 INFO:teuthology.orchestra.run.smithi170.stdout:2309 punch from 0xc51800 to 0xc5f000, (0xd800 bytes) 2024-09-21T23:15:54.783 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:54.789 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:54.788+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:54.803 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:54.809 INFO:teuthology.orchestra.run.smithi170.stdout:2310 write 0xb071400 thru 0xb074ad9 (0x36da bytes) 2024-09-21T23:15:54.811 INFO:teuthology.orchestra.run.smithi170.stdout:2311 punch from 0x55a3000 to 0x55a4200, (0x1200 bytes) 2024-09-21T23:15:54.825 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:54.825 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:54.830 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:54.831 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:54.832 INFO:teuthology.orchestra.run.smithi170.stdout:2312 write 0xc692800 thru 0xc69445d (0x1c5e bytes) 2024-09-21T23:15:54.852 INFO:teuthology.orchestra.run.smithi170.stdout:2313 write 0xc6ccc00 thru 0xc6cf3d1 (0x27d2 bytes) 2024-09-21T23:15:54.856 INFO:teuthology.orchestra.run.smithi170.stdout:2314 trunc from 0xc6cf400 to 0xc629c00 2024-09-21T23:15:54.868 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:54.868 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:54.880 INFO:teuthology.orchestra.run.smithi170.stdout:2315 punch from 0x3097c00 to 0x3098200, (0x600 bytes) 2024-09-21T23:15:54.881 INFO:teuthology.orchestra.run.smithi170.stdout:2317 punch from 0x73f400 to 0x740e00, (0x1a00 bytes) 2024-09-21T23:15:54.882 INFO:teuthology.orchestra.run.smithi170.stdout:2321 write 0x6fa9000 thru 0x6fafdea (0x6deb bytes) 2024-09-21T23:15:54.882 INFO:teuthology.orchestra.run.smithi170.stdout:2322 read 0x604cc00 thru 0x6058769 (0xbb6a bytes) 2024-09-21T23:15:54.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:54.890 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:54.883+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:54.898 INFO:teuthology.orchestra.run.smithi170.stdout:2323 trunc from 0xc629c00 to 0x64ec800 2024-09-21T23:15:54.932 INFO:teuthology.orchestra.run.smithi170.stdout:2326 trunc from 0x64ec800 to 0x5ff3600 2024-09-21T23:15:54.956 INFO:teuthology.orchestra.run.smithi170.stdout:2327 read 0x4062800 thru 0x406c901 (0xa102 bytes) 2024-09-21T23:15:54.957 INFO:teuthology.orchestra.run.smithi170.stdout:2328 read 0x41f5600 thru 0x42032a3 (0xdca4 bytes) 2024-09-21T23:15:54.966 INFO:teuthology.orchestra.run.smithi170.stdout:2330 trunc from 0x5ff3600 to 0x9051200 2024-09-21T23:15:54.985 INFO:teuthology.orchestra.run.smithi170.stdout:2332 read 0x60d2000 thru 0x60d662e (0x462f bytes) 2024-09-21T23:15:54.991 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:54.987+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:54.995 INFO:teuthology.orchestra.run.smithi170.stdout:2335 trunc from 0x9051200 to 0x543d200 2024-09-21T23:15:55.020 INFO:teuthology.orchestra.run.smithi170.stdout:2336 punch from 0x4ecf800 to 0x4edee00, (0xf600 bytes) 2024-09-21T23:15:55.021 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:55.021 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:55.036 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:55.053 INFO:teuthology.orchestra.run.smithi170.stdout:2337 write 0x9e1fa00 thru 0x9e254e5 (0x5ae6 bytes) 2024-09-21T23:15:55.054 INFO:teuthology.orchestra.run.smithi170.stdout:2338 read 0x102600 thru 0x10cec4 (0xa8c5 bytes) 2024-09-21T23:15:55.055 INFO:teuthology.orchestra.run.smithi170.stdout:2339 punch from 0x93bac00 to 0x93c4c00, (0xa000 bytes) 2024-09-21T23:15:55.082 INFO:teuthology.orchestra.run.smithi170.stdout:2341 write 0xc7de800 thru 0xc7eaee8 (0xc6e9 bytes) 2024-09-21T23:15:55.084 INFO:teuthology.orchestra.run.smithi170.stdout:2342 trunc from 0xc7eb000 to 0xc8d5600 2024-09-21T23:15:55.122 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:55.119+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:55.159 INFO:teuthology.orchestra.run.smithi170.stdout:2345 trunc from 0xc8d5600 to 0x31a00 2024-09-21T23:15:55.194 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:55.187+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:55.207 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:55.207 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:55.222 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:55.264 INFO:teuthology.orchestra.run.smithi170.stdout:2346 write 0xc80d200 thru 0xc8176bc (0xa4bd bytes) 2024-09-21T23:15:55.265 INFO:teuthology.orchestra.run.smithi170.stdout:2349 write 0x8ce9c00 thru 0x8cf54c3 (0xb8c4 bytes) 2024-09-21T23:15:55.266 INFO:teuthology.orchestra.run.smithi170.stdout:2353 read 0x9c0c000 thru 0x9c0efc7 (0x2fc8 bytes) 2024-09-21T23:15:55.266 INFO:teuthology.orchestra.run.smithi170.stdout:2354 read 0x54b200 thru 0x5534ff (0x8300 bytes) 2024-09-21T23:15:55.267 INFO:teuthology.orchestra.run.smithi170.stdout:2356 write 0xaf96c00 thru 0xaf99843 (0x2c44 bytes) 2024-09-21T23:15:55.267 INFO:teuthology.orchestra.run.smithi170.stdout:2358 clone 13 order 20 su 65536 sc 10 2024-09-21T23:15:55.295 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:55.292+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:55.383 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:55.383 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:55.395 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:55.391+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:55.396 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:55.495 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:55.491+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:55.559 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:55.560 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:55.573 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:55.615 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:55.611+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:55.697 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:55.691+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:55.708 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:55.708 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:55.718 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:15:55.716+0000 7f9740dfd7c0 -1 Falling back to public interface 2024-09-21T23:15:55.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:55.797 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:55.796+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:55.847 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:55.847 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:55.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:55.898 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:55.895+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:55.999 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:55.996+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:56.029 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:56.029 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:56.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:56.100 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:56.095+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:56.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:15:56.201 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:56.200+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:56.208 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:56.208 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:56.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:56.312 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:56.312+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:56.384 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:56.384 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:56.396 INFO:teuthology.orchestra.run.smithi170.stdout:flattening image image_client.0-clone12 2024-09-21T23:15:56.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:56.440 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:56.440+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:56.503 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:56.499+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:56.561 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:56.561 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:56.577 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:56.604 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:56.604+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:56.704 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:56.699+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:56.710 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:56.710 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:56.725 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:56.806 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:56.804+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:56.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:15:56.896 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:56.896 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:56.907 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:56.903+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:56.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:56.975 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:56.993 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:56.993 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:57.008 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:57.003+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:57.010 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:57.030 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:57.030 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:57.050 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:57.050 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:57.071 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:57.071 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:57.075 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:57.075 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:57.087 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.863106619000007, 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:15:57.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.350701336, 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.095 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:15:57.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:15:57.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:15:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.350636433000005, 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:15:57.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:15:57.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:15:57.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:15:57.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:15:57.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.108 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.350489994, 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.109 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:15:57.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.350420878999998, 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.115 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:15:57.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:15:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.349808554000006, 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.121 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:15:57.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.126 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.349604149000001, 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:15:57.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:15:57.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:57.132 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.349561363000007, 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:57.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:15:57.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.958603074999999, 2024-09-21T23:15:57.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:15:57.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.958516564999996, 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.144 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:15:57.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.958439675999998, 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.150 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:15:57.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:15:57.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.958234836000003, 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.156 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:15:57.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:15:57.161 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.958146553999995, 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:15:57.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:15:57.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.167 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.958015725999999, 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:15:57.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:15:57.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.957970072999998, 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:15:57.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:15:57.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.957931646000006, 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:15:57.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:15:57.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:57.185 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.957802184000002, 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:15:57.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.191 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.957780541999995, 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:15:57.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.197 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.957760497999999, 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:15:57.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:15:57.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.957676505999999, 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:15:57.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.209 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:57.210 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-21T23:15:57.212 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:15:57.212 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:15:57.212 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:57.207+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:57.311 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:57.307+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:57.314 INFO:teuthology.orchestra.run.smithi170.stdout:2361 read 0x5030800 thru 0x50336d9 (0x2eda bytes) 2024-09-21T23:15:57.314 INFO:teuthology.orchestra.run.smithi170.stdout:2363 write 0x4206000 thru 0x4208773 (0x2774 bytes) 2024-09-21T23:15:57.315 INFO:teuthology.orchestra.run.smithi170.stdout:2365 write 0x5c99e00 thru 0x5ca5441 (0xb642 bytes) 2024-09-21T23:15:57.323 INFO:teuthology.orchestra.run.smithi170.stdout:2366 trunc from 0xc817800 to 0xdad0a00 2024-09-21T23:15:57.362 INFO:teuthology.orchestra.run.smithi170.stdout:2367 read 0x8875a00 thru 0x888122a (0xb82b bytes) 2024-09-21T23:15:57.366 INFO:teuthology.orchestra.run.smithi170.stdout:2372 write 0x1071000 thru 0x107e17c (0xd17d bytes) 2024-09-21T23:15:57.366 INFO:teuthology.orchestra.run.smithi170.stdout:2373 read 0x883ca00 thru 0x88451b6 (0x87b7 bytes) 2024-09-21T23:15:57.367 INFO:teuthology.orchestra.run.smithi170.stdout:2374 write 0x36d9800 thru 0x36ddd0e (0x450f bytes) 2024-09-21T23:15:57.367 INFO:teuthology.orchestra.run.smithi170.stdout:2379 read 0x14e3a00 thru 0x14e3d14 (0x315 bytes) 2024-09-21T23:15:57.367 INFO:teuthology.orchestra.run.smithi170.stdout:2381 read 0x5e0ec00 thru 0x5e101e5 (0x15e6 bytes) 2024-09-21T23:15:57.367 INFO:teuthology.orchestra.run.smithi170.stdout:2385 read 0xca01800 thru 0xca0ec46 (0xd447 bytes) 2024-09-21T23:15:57.367 INFO:teuthology.orchestra.run.smithi170.stdout:2386 punch from 0xc1c8800 to 0xc1d6c00, (0xe400 bytes) 2024-09-21T23:15:57.368 INFO:teuthology.orchestra.run.smithi170.stdout:2387 read 0x1133200 thru 0x113fd28 (0xcb29 bytes) 2024-09-21T23:15:57.369 INFO:teuthology.orchestra.run.smithi170.stdout:2392 punch from 0x3f09400 to 0x3f10e00, (0x7a00 bytes) 2024-09-21T23:15:57.369 INFO:teuthology.orchestra.run.smithi170.stdout:2393 write 0x33a400 thru 0x34a2b4 (0xfeb5 bytes) 2024-09-21T23:15:57.371 INFO:teuthology.orchestra.run.smithi170.stdout:2395 read 0xb754200 thru 0xb756290 (0x2091 bytes) 2024-09-21T23:15:57.371 INFO:teuthology.orchestra.run.smithi170.stdout:2399 read 0x2dc6400 thru 0x2dc8bb5 (0x27b6 bytes) 2024-09-21T23:15:57.371 INFO:teuthology.orchestra.run.smithi170.stdout:2401 write 0xa13f000 thru 0xa145036 (0x6037 bytes) 2024-09-21T23:15:57.398 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:57.398 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:57.399 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:57.399 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:57.411 INFO:teuthology.orchestra.run.smithi170.stdout:2402 trunc from 0xdad0a00 to 0x6103200 2024-09-21T23:15:57.412 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:57.412+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:57.414 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:15:57.473 INFO:teuthology.orchestra.run.smithi170.stdout:2403 read 0x318be00 thru 0x318fe98 (0x4099 bytes) 2024-09-21T23:15:57.477 INFO:teuthology.orchestra.run.smithi170.stdout:2404 punch from 0x41c7800 to 0x41ca600, (0x2e00 bytes) 2024-09-21T23:15:57.477 INFO:teuthology.orchestra.run.smithi170.stdout:2407 punch from 0x46dc800 to 0x46e2a00, (0x6200 bytes) 2024-09-21T23:15:57.491 INFO:teuthology.orchestra.run.smithi170.stdout:2410 trunc from 0x6103200 to 0xc755600 2024-09-21T23:15:57.513 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:57.512+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:57.516 INFO:teuthology.orchestra.run.smithi170.stdout:2411 write 0x756e400 thru 0x75747c8 (0x63c9 bytes) 2024-09-21T23:15:57.547 INFO:teuthology.orchestra.run.smithi170.stdout:2412 trunc from 0xc755600 to 0x4fcd400 2024-09-21T23:15:57.584 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:57.584 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:57.584 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:57.584 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:57.584 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:57.599 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:15:57.646 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:57.643+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:57.654 INFO:teuthology.orchestra.run.smithi170.stdout:2413 write 0xce16a00 thru 0xce20ad6 (0xa0d7 bytes) 2024-09-21T23:15:57.656 INFO:teuthology.orchestra.run.smithi170.stdout:2414 write 0xc4b9c00 thru 0xc4bc4a3 (0x28a4 bytes) 2024-09-21T23:15:57.656 INFO:teuthology.orchestra.run.smithi170.stdout:2417 read 0x3da00 thru 0x4c3c8 (0xe9c9 bytes) 2024-09-21T23:15:57.658 INFO:teuthology.orchestra.run.smithi170.stdout:2418 read 0x6012600 thru 0x6017948 (0x5349 bytes) 2024-09-21T23:15:57.658 INFO:teuthology.orchestra.run.smithi170.stdout:2419 write 0x2755a00 thru 0x2758408 (0x2a09 bytes) 2024-09-21T23:15:57.658 INFO:teuthology.orchestra.run.smithi170.stdout:2420 write 0xa32f800 thru 0xa33ce2e (0xd62f bytes) 2024-09-21T23:15:57.667 INFO:teuthology.orchestra.run.smithi170.stdout:2421 trunc from 0xce20c00 to 0xe10c200 2024-09-21T23:15:57.698 INFO:teuthology.orchestra.run.smithi170.stdout:2422 read 0xd4cb800 thru 0xd4cb945 (0x146 bytes) 2024-09-21T23:15:57.702 INFO:teuthology.orchestra.run.smithi170.stdout:2423 write 0x7206800 thru 0x720b98c (0x518d bytes) 2024-09-21T23:15:57.703 INFO:teuthology.orchestra.run.smithi170.stdout:2424 write 0xa8e5c00 thru 0xa8f2806 (0xcc07 bytes) 2024-09-21T23:15:57.703 INFO:teuthology.orchestra.run.smithi170.stdout:2425 write 0x6c64a00 thru 0x6c64eba (0x4bb bytes) 2024-09-21T23:15:57.703 INFO:teuthology.orchestra.run.smithi170.stdout:2426 write 0xd541800 thru 0xd550c63 (0xf464 bytes) 2024-09-21T23:15:57.704 INFO:teuthology.orchestra.run.smithi170.stdout:2428 trunc from 0xe10c200 to 0xd9e3800 2024-09-21T23:15:57.714 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:57.711+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:57.735 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.778158758, 2024-09-21T23:15:57.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:15:57.737 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.737 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.737 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:15:57.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:57.738 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:15:57.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.738 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:15:57.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.777891774, 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.744 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:15:57.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:15:57.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3357 3.11 3:8fb7664f:::rbd_data.10f1cfc9677.0000000000000015:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.558937930999999, 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049070639999999999, 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3357 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173029+0000", 2024-09-21T23:15:57.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.494e-06 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173037+0000", 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173060+0000", 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717999999999998e-05 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173074+0000", 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4671000000000001e-05 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204211+0000", 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031136891999999999 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204347+0000", 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013595400000000001 2024-09-21T23:15:57.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222098+0000", 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017750717999999999 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3418 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550919167, 2024-09-21T23:15:57.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054653000000003, 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3418 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:57.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181047+0000", 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6899999999999996e-07 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181057+0000", 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0223e-05 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181064+0000", 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9949999999999999e-06 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181069+0000", 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4160000000000003e-06 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221944+0000", 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040874684000000001 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222101+0000", 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015656599999999999 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3419 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550888173000001, 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041067436999999998, 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3419 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.760 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181080+0000", 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.402e-06 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181087+0000", 2024-09-21T23:15:57.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181092+0000", 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5720000000000002e-06 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222136+0000", 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041044316999999997 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222144+0000", 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9349999999999994e-06 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3420 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.55086575, 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041092287999999998, 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.764 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3420 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181100+0000", 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8500000000000001e-07 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181102+0000", 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2879999999999999e-06 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181109+0000", 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181115+0000", 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7819999999999999e-06 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222181+0000", 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041065989999999997 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222192+0000", 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.076e-05 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.768 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3421 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550843252, 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041102946000000001, 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3421 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:15:57.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181125+0000", 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5390000000000002e-06 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181132+0000", 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6610000000000003e-06 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181137+0000", 2024-09-21T23:15:57.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2429999999999996e-06 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222217+0000", 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041080113000000001 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222225+0000", 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9049999999999997e-06 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3422 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550820671, 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041150084000000003, 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3422 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.773 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181145+0000", 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181148+0000", 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.57e-06 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181154+0000", 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9909999999999999e-06 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181160+0000", 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222287+0000", 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041126708999999997 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222295+0000", 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6799999999999993e-06 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.776 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3424 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550797800000002, 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041160906999999997, 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.777 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3424 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181168+0000", 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181170+0000", 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181178+0000", 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181183+0000", 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3410000000000002e-06 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222321+0000", 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041137765 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222328+0000", 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6550000000000004e-06 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3425 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550774945000001, 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041445086999999999, 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.781 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3425 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:15:57.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181191+0000", 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999999e-07 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181193+0000", 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6800000000000002e-06 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181200+0000", 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6760000000000001e-06 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181205+0000", 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.383e-06 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222627+0000", 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041421342 2024-09-21T23:15:57.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222635+0000", 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6130000000000004e-06 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3509 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550394301000001, 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040950634, 2024-09-21T23:15:57.785 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3509 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181574+0000", 2024-09-21T23:15:57.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.199e-06 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181580+0000", 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0560000000000001e-06 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181585+0000", 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1669999999999998e-06 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222516+0000", 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931568000000002 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222521+0000", 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3510 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:57.789 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550371559999999, 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040943715999999998, 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3510 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.32e-07 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181596+0000", 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.339e-06 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181603+0000", 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181608+0000", 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222534+0000", 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040925221999999997 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222537+0000", 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.793 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3567 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.550008426000002, 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040956498000000001, 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3567 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7899999999999999e-07 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181959+0000", 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1830000000000001e-06 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181966+0000", 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3119999999999999e-06 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181973+0000", 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:15:57.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222902+0000", 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928961999999999 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222913+0000", 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1552000000000001e-05 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:499 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9520537789999999, 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041124135999999999, 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 499 2024-09-21T23:15:57.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779913+0000", 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779918+0000", 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779938+0000", 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9553000000000001e-05 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779957+0000", 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820711+0000", 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040754603 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820848+0000", 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013633699999999999 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821036+0000", 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018796899999999999 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:500 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9519886389999996, 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041461570000000003, 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 500 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779978+0000", 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1969999999999999e-06 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779983+0000", 2024-09-21T23:15:57.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780045+0000", 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2082000000000006e-05 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780087+0000", 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1863000000000002e-05 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821117+0000", 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041030076999999998 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821247+0000", 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012996199999999999 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821438+0000", 2024-09-21T23:15:57.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019079899999999999 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:501 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.951794907, 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042023224999999997, 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 501 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:15:57.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780172+0000", 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780184+0000", 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1039999999999999e-05 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780215+0000", 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1006000000000001e-05 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780235+0000", 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0483999999999999e-05 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821553+0000", 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041318311000000003 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821848+0000", 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029481199999999998 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822193+0000", 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034536499999999998 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:506 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9516963540000001, 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041920526, 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 506 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:15:57.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780270+0000", 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3370000000000001e-06 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780278+0000", 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8730000000000007e-06 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780300+0000", 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2183000000000001e-05 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780316+0000", 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5413e-05 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821521+0000", 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041205506000000003 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821802+0000", 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028134500000000001 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822189+0000", 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038686900000000003 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.816 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:507 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:15:57.817 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:15:57.817 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9516264880000005, 2024-09-21T23:15:57.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042296304, 2024-09-21T23:15:57.817 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.817 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.817 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 507 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780340+0000", 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.31e-06 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780347+0000", 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780367+0000", 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9466e-05 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780382+0000", 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5274e-05 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822269+0000", 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041886592 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822418+0000", 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014924299999999999 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822635+0000", 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021715499999999999 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:508 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9514860350000003, 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042742988000000003, 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:57.822 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 508 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780481+0000", 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4869999999999999e-06 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780489+0000", 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1200000000000002e-06 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781280+0000", 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000791583 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781307+0000", 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6486999999999999e-05 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822728+0000", 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041420917000000002 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822937+0000", 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000209593 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823222+0000", 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000284801 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:509 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.9502405700000001, 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041990254999999997, 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 509 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:57.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781726+0000", 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2759999999999999e-06 2024-09-21T23:15:57.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781744+0000", 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8377e-05 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781767+0000", 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3006999999999998e-05 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781780+0000", 2024-09-21T23:15:57.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2393e-05 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823275+0000", 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041494879999999998 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823485+0000", 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021036300000000001 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:57.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823715+0000", 2024-09-21T23:15:57.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022995900000000001 2024-09-21T23:15:57.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.831 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:57.831 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:57.831 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:57.832 INFO:teuthology.orchestra.run.smithi170.stdout:2429 trunc from 0xd9e3800 to 0x9555c00 2024-09-21T23:15:57.832 INFO:teuthology.orchestra.run.smithi170.stdout:2432 trunc from 0x9555c00 to 0xc00be00 2024-09-21T23:15:57.833 INFO:teuthology.orchestra.run.smithi170.stdout:2433 write 0x4f1a000 thru 0x4f268a0 (0xc8a1 bytes) 2024-09-21T23:15:57.833 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:15:57.834 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:57.827+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:57.834 INFO:teuthology.orchestra.run.smithi170.stdout:2434 trunc from 0xc00be00 to 0xe3efa00 2024-09-21T23:15:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:2438 write 0x21c6200 thru 0x21c6ed2 (0xcd3 bytes) 2024-09-21T23:15:57.860 INFO:teuthology.orchestra.run.smithi170.stdout:2439 read 0xbdeca00 thru 0xbdede18 (0x1419 bytes) 2024-09-21T23:15:57.860 INFO:teuthology.orchestra.run.smithi170.stdout:2440 read 0x7d0600 thru 0x7dd236 (0xcc37 bytes) 2024-09-21T23:15:57.861 INFO:teuthology.orchestra.run.smithi170.stdout:2442 write 0x4b82c00 thru 0x4b8a80f (0x7c10 bytes) 2024-09-21T23:15:57.862 INFO:teuthology.orchestra.run.smithi170.stdout:2443 read 0x31b4600 thru 0x31bc165 (0x7b66 bytes) 2024-09-21T23:15:57.863 INFO:teuthology.orchestra.run.smithi170.stdout:2446 read 0xde29000 thru 0xde31c9d (0x8c9e bytes) 2024-09-21T23:15:57.863 INFO:teuthology.orchestra.run.smithi170.stdout:2447 read 0x25d2e00 thru 0x25d8dbc (0x5fbd bytes) 2024-09-21T23:15:57.863 INFO:teuthology.orchestra.run.smithi170.stdout:2448 write 0x39d8000 thru 0x39dbb43 (0x3b44 bytes) 2024-09-21T23:15:57.863 INFO:teuthology.orchestra.run.smithi170.stdout:2451 write 0xbcd9200 thru 0xbce81b7 (0xefb8 bytes) 2024-09-21T23:15:57.865 INFO:teuthology.orchestra.run.smithi170.stdout:2455 trunc from 0xe3efa00 to 0xd83a200 2024-09-21T23:15:57.892 INFO:teuthology.orchestra.run.smithi170.stdout:2456 read 0x5f2d600 thru 0x5f31a67 (0x4468 bytes) 2024-09-21T23:15:57.894 INFO:teuthology.orchestra.run.smithi170.stdout:2457 read 0x317ee00 thru 0x317f667 (0x868 bytes) 2024-09-21T23:15:57.894 INFO:teuthology.orchestra.run.smithi170.stdout:2458 read 0x30b8000 thru 0x30c7f24 (0xff25 bytes) 2024-09-21T23:15:57.895 INFO:teuthology.orchestra.run.smithi170.stdout:2461 write 0x653c400 thru 0x6546f25 (0xab26 bytes) 2024-09-21T23:15:57.895 INFO:teuthology.orchestra.run.smithi170.stdout:2462 write 0x571ec00 thru 0x5723726 (0x4b27 bytes) 2024-09-21T23:15:57.906 INFO:teuthology.orchestra.run.smithi170.stdout:2464 trunc from 0xd83a200 to 0x78de600 2024-09-21T23:15:57.933 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:57.927+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:57.983 INFO:teuthology.orchestra.run.smithi170.stdout:2469 trunc from 0x78de600 to 0x12fdc00 2024-09-21T23:15:58.013 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:58.013 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:58.013 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:15:58.013 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:58.031 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:15:58.034 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:58.032+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:58.070 INFO:teuthology.orchestra.run.smithi170.stdout:2472 trunc from 0x12fdc00 to 0x463200 2024-09-21T23:15:58.135 INFO:teuthology.orchestra.run.smithi170.stdout:2474 punch from 0x6d600 to 0x72200, (0x4c00 bytes) 2024-09-21T23:15:58.135 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:58.132+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:58.136 INFO:teuthology.orchestra.run.smithi170.stdout:2476 read 0x296600 thru 0x29bb9c (0x559d bytes) 2024-09-21T23:15:58.182 INFO:teuthology.orchestra.run.smithi170.stdout:2477 write 0xa249c00 thru 0xa252846 (0x8c47 bytes) 2024-09-21T23:15:58.182 INFO:teuthology.orchestra.run.smithi170.stdout:2479 write 0xc88a00 thru 0xc97810 (0xee11 bytes) 2024-09-21T23:15:58.183 INFO:teuthology.orchestra.run.smithi170.stdout:2480 read 0x4332800 thru 0x433b5f5 (0x8df6 bytes) 2024-09-21T23:15:58.183 INFO:teuthology.orchestra.run.smithi170.stdout:2483 write 0xeaf800 thru 0xeafcd2 (0x4d3 bytes) 2024-09-21T23:15:58.183 INFO:teuthology.orchestra.run.smithi170.stdout:2488 punch from 0x7ca7200 to 0x7cad000, (0x5e00 bytes) 2024-09-21T23:15:58.184 INFO:teuthology.orchestra.run.smithi170.stdout:2490 write 0x904d800 thru 0x9053004 (0x5805 bytes) 2024-09-21T23:15:58.210 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:58.210 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:15:58.210 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:15:58.210 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:15:58.210 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:58.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:15:58.225 INFO:teuthology.orchestra.run.smithi170.stdout:2492 write 0xedc4400 thru 0xedc9ca7 (0x58a8 bytes) 2024-09-21T23:15:58.227 INFO:teuthology.orchestra.run.smithi170.stdout:2495 punch from 0xc37f200 to 0xc38a000, (0xae00 bytes) 2024-09-21T23:15:58.228 INFO:teuthology.orchestra.run.smithi170.stdout:2496 write 0x8480a00 thru 0x8488557 (0x7b58 bytes) 2024-09-21T23:15:58.228 INFO:teuthology.orchestra.run.smithi170.stdout:2502 write 0x91c5400 thru 0x91c63c9 (0xfca bytes) 2024-09-21T23:15:58.236 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:58.236+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:58.247 INFO:teuthology.orchestra.run.smithi170.stdout:2503 trunc from 0xedc9e00 to 0x4d93a00 2024-09-21T23:15:58.284 INFO:teuthology.orchestra.run.smithi170.stdout:2504 write 0x1462800 thru 0x1464842 (0x2043 bytes) 2024-09-21T23:15:58.330 INFO:teuthology.orchestra.run.smithi170.stdout:2506 write 0xb1e2c00 thru 0xb1e4903 (0x1d04 bytes) 2024-09-21T23:15:58.337 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:58.336+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3198 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 2244608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.167353699, 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034480396000000003, 2024-09-21T23:15:58.357 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3198 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185427+0000", 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8400000000000003e-07 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185430+0000", 2024-09-21T23:15:58.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.275e-06 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185439+0000", 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6500000000000002e-06 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185454+0000", 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4778e-05 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213252+0000", 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027798665 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213316+0000", 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3701999999999994e-05 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219907+0000", 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065904420000000002 2024-09-21T23:15:58.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3349 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2179072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.164314954999998, 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029976903999999999, 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3349 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188467+0000", 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1390000000000002e-06 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188473+0000", 2024-09-21T23:15:58.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0159999999999997e-06 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188478+0000", 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5480000000000003e-06 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218328+0000", 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029849969000000001 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218367+0000", 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9474999999999997e-05 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218442+0000", 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4418999999999993e-05 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.366 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3350 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2260992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.164291142, 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030043218999999999, 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3350 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:15:58.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188492+0000", 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2419999999999999e-06 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188499+0000", 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0670000000000003e-06 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188504+0000", 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9699999999999998e-06 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218457+0000", 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029953832999999999 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218493+0000", 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5216000000000003e-05 2024-09-21T23:15:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218532+0000", 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9267000000000002e-05 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3351 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2342912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.164269066999999, 2024-09-21T23:15:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030166889999999998, 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3351 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188514+0000", 2024-09-21T23:15:58.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5210000000000001e-06 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188520+0000", 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4799999999999998e-06 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188526+0000", 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3179999999999997e-06 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218555+0000", 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029059 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218613+0000", 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8084999999999997e-05 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218678+0000", 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5021e-05 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3352 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2424832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.164245548, 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.03030147, 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.376 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3352 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:15:58.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188535+0000", 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188540+0000", 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6790000000000002e-06 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188547+0000", 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2439999999999998e-06 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188551+0000", 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7310000000000001e-06 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218703+0000", 2024-09-21T23:15:58.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030151305999999999 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218768+0000", 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5512999999999994e-05 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218836+0000", 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7550000000000002e-05 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3353 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2506752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.164219222, 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030431269, 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3353 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.381 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:15:58.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8599999999999998e-07 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188564+0000", 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4229999999999999e-06 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188570+0000", 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1199999999999999e-06 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188577+0000", 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.221e-06 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218860+0000", 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030282878999999999 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218924+0000", 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3948999999999994e-05 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218992+0000", 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190999999999998e-05 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3354 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2588672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.164196799999999, 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030565452, 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.385 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3354 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188586+0000", 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1919999999999999e-06 2024-09-21T23:15:58.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188592+0000", 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2380000000000002e-06 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188597+0000", 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0819999999999998e-06 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219018+0000", 2024-09-21T23:15:58.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030420823 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219084+0000", 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6513000000000005e-05 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219149+0000", 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4187000000000001e-05 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3460 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.163229898000001, 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029906078999999999, 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3460 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:15:58.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189553+0000", 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3010000000000001e-06 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189559+0000", 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189564+0000", 2024-09-21T23:15:58.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1819999999999997e-06 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219323+0000", 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029758623000000001 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219391+0000", 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8330000000000005e-05 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219456+0000", 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4747999999999997e-05 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3462 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.163184781999998, 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030022725, 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3462 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:15:58.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189596+0000", 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189598+0000", 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8260000000000001e-06 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189605+0000", 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6919999999999996e-06 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189611+0000", 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2360000000000001e-06 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219480+0000", 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029868714000000001 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219547+0000", 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7268999999999996e-05 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219618+0000", 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0508999999999999e-05 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3469 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.398 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162712152000001, 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029238999999999, 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3469 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190069+0000", 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2500000000000004e-07 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190071+0000", 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9520000000000001e-06 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190075+0000", 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8420000000000004e-06 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190080+0000", 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4800000000000003e-06 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219961+0000", 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029880803000000001 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220030+0000", 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8856999999999995e-05 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220097+0000", 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7379999999999995e-05 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3470 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162693893, 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030180927999999999, 2024-09-21T23:15:58.403 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3470 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190087+0000", 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5030000000000001e-06 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190090+0000", 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.278e-06 2024-09-21T23:15:58.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190095+0000", 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8910000000000003e-06 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190100+0000", 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9710000000000003e-06 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220128+0000", 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030028195000000001 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220198+0000", 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0405000000000001e-05 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220267+0000", 2024-09-21T23:15:58.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8684999999999997e-05 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3472 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162653814999999, 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030301255999999999, 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.408 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3472 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190127+0000", 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.238e-06 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190130+0000", 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190135+0000", 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0270000000000003e-06 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190140+0000", 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8609999999999997e-06 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220297+0000", 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030157498000000001 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220368+0000", 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0811999999999999e-05 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220427+0000", 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9234999999999998e-05 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3484 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3538944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:58.412 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162575628999999, 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030347946000000001, 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3484 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190205+0000", 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.13e-06 2024-09-21T23:15:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190222+0000", 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6909999999999999e-05 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190229+0000", 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2979999999999997e-06 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190234+0000", 2024-09-21T23:15:58.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7130000000000002e-06 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220443+0000", 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030209037000000001 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220490+0000", 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6198000000000001e-05 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220552+0000", 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.266e-05 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3486 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162518029000001, 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030430263999999999, 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3486 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:58.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190264+0000", 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.007e-06 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190270+0000", 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190275+0000", 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220569+0000", 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030294237000000002 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220625+0000", 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5902000000000001e-05 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220692+0000", 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6796000000000002e-05 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.421 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3487 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162497854000001, 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030517598, 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3487 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190284+0000", 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.66e-06 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190287+0000", 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9359999999999999e-06 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190292+0000", 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5219999999999998e-06 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190297+0000", 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0900000000000004e-06 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220721+0000", 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030423823999999999 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220759+0000", 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7962e-05 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220800+0000", 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0604e-05 2024-09-21T23:15:58.425 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3489 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3784704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162456911, 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030596542000000001, 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.426 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3489 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190324+0000", 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190327+0000", 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6180000000000002e-06 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190333+0000", 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7259999999999999e-06 2024-09-21T23:15:58.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190338+0000", 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1680000000000003e-06 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220825+0000", 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030487104000000001 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220863+0000", 2024-09-21T23:15:58.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8563000000000003e-05 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220920+0000", 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6051999999999998e-05 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3490 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3866624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:58.430 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162435184, 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030740849000000001, 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3490 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:58.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190346+0000", 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5859999999999999e-06 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190349+0000", 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190354+0000", 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1950000000000002e-06 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190359+0000", 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9819999999999999e-06 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220944+0000", 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030585309000000001 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221006+0000", 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1802000000000002e-05 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221086+0000", 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9314999999999995e-05 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3491 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162399819000001, 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030871663000000001, 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.435 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3491 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190382+0000", 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.522e-06 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190384+0000", 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:15:58.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190391+0000", 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2899999999999999e-06 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190396+0000", 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9570000000000001e-06 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221112+0000", 2024-09-21T23:15:58.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030716753999999999 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221179+0000", 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6122000000000002e-05 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221252+0000", 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3152999999999996e-05 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.439 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3493 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162356136, 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030983984999999999, 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3493 2024-09-21T23:15:58.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2800000000000002e-07 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190427+0000", 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4660000000000002e-06 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190433+0000", 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3110000000000002e-06 2024-09-21T23:15:58.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190438+0000", 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7539999999999997e-06 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221282+0000", 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030844399000000002 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221345+0000", 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3145999999999995e-05 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221408+0000", 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2481e-05 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3495 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:58.444 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 17.162313394000002, 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031095845, 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3495 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:58.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190468+0000", 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:15:58.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190470+0000", 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5509999999999998e-06 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190475+0000", 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190481+0000", 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4389999999999999e-06 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221436+0000", 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030955580999999999 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221501+0000", 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5030999999999995e-05 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221562+0000", 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1061000000000004e-05 2024-09-21T23:15:58.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.449 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:15:58.449 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:15:58.449 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:15:58.451 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:58.443+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:58.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2507 write 0xe3d9a00 thru 0xe3da532 (0xb33 bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2508 read 0x4f15e00 thru 0x4f1fb8a (0x9d8b bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2510 write 0x439e200 thru 0x43a9be6 (0xb9e7 bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2511 read 0xe11b000 thru 0xe120bd8 (0x5bd9 bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2512 read 0x664ce00 thru 0x6653013 (0x6214 bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2515 write 0x369d600 thru 0x36a7f92 (0xa993 bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2516 write 0x24c1000 thru 0x24c7931 (0x6932 bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2518 punch from 0xcc10600 to 0xcc18000, (0x7a00 bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2519 read 0x717fc00 thru 0x71884ce (0x88cf bytes) 2024-09-21T23:15:58.453 INFO:teuthology.orchestra.run.smithi170.stdout:2520 write 0x2988c00 thru 0x2991396 (0x8797 bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2522 read 0xd702400 thru 0xd707e9f (0x5aa0 bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2523 write 0x3f02200 thru 0x3f0e2ad (0xc0ae bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2525 read 0xc04c200 thru 0xc05afd0 (0xedd1 bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2527 write 0x26cd200 thru 0x26cf37f (0x2180 bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2528 trunc from 0xe3da600 to 0xb354a00 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2532 trunc from 0xb354a00 to 0xc31bc00 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2534 punch from 0x8fc1c00 to 0x8fc6200, (0x4600 bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2536 write 0x7f2f600 thru 0x7f2fb43 (0x544 bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2537 read 0x2daa800 thru 0x2db2741 (0x7f42 bytes) 2024-09-21T23:15:58.454 INFO:teuthology.orchestra.run.smithi170.stdout:2539 read 0xb11e00 thru 0xb1aefa (0x90fb bytes) 2024-09-21T23:15:58.464 INFO:teuthology.orchestra.run.smithi170.stdout:2541 trunc from 0xc31bc00 to 0x251aa00 2024-09-21T23:15:58.503 INFO:teuthology.orchestra.run.smithi170.stdout:2544 write 0x1893000 thru 0x189577a (0x277b bytes) 2024-09-21T23:15:58.507 INFO:teuthology.orchestra.run.smithi170.stdout:2545 punch from 0x1088c00 to 0x1091000, (0x8400 bytes) 2024-09-21T23:15:58.507 INFO:teuthology.orchestra.run.smithi170.stdout:2546 write 0xdb6200 thru 0xdbbe17 (0x5c18 bytes) 2024-09-21T23:15:58.507 INFO:teuthology.orchestra.run.smithi170.stdout:2548 write 0xbb1c00 thru 0xbb9859 (0x7c5a bytes) 2024-09-21T23:15:58.508 INFO:teuthology.orchestra.run.smithi170.stdout:2549 punch from 0x1cf8000 to 0x1d02600, (0xa600 bytes) 2024-09-21T23:15:58.508 INFO:teuthology.orchestra.run.smithi170.stdout:2550 punch from 0x20bb800 to 0x20c6600, (0xae00 bytes) 2024-09-21T23:15:58.512 INFO:teuthology.orchestra.run.smithi170.stdout:2556 trunc from 0x251aa00 to 0x465c000 2024-09-21T23:15:58.543 INFO:teuthology.orchestra.run.smithi170.stdout:2557 punch from 0x25c1a00 to 0x25c2e00, (0x1400 bytes) 2024-09-21T23:15:58.545 INFO:teuthology.orchestra.run.smithi170.stdout:2558 read 0x4c4e00 thru 0x4cdf36 (0x9137 bytes) 2024-09-21T23:15:58.550 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:58.548+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:58.568 INFO:teuthology.orchestra.run.smithi170.stdout:2560 write 0x823fc00 thru 0x824eba7 (0xefa8 bytes) 2024-09-21T23:15:58.570 INFO:teuthology.orchestra.run.smithi170.stdout:2562 write 0x73c6000 thru 0x73c6633 (0x634 bytes) 2024-09-21T23:15:58.571 INFO:teuthology.orchestra.run.smithi170.stdout:2566 punch from 0xd54000 to 0xd63600, (0xf600 bytes) 2024-09-21T23:15:58.594 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:58.594 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:58.594 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:58.595 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:58.600 INFO:teuthology.orchestra.run.smithi170.stdout:2569 write 0xb07ca00 thru 0xb08c43d (0xfa3e bytes) 2024-09-21T23:15:58.602 INFO:teuthology.orchestra.run.smithi170.stdout:2570 read 0x9d39e00 thru 0x9d49941 (0xfb42 bytes) 2024-09-21T23:15:58.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:15:58.613 INFO:teuthology.orchestra.run.smithi170.stdout:2571 trunc from 0xb08c600 to 0x4bcfc00 2024-09-21T23:15:58.651 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:15:58.647+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:15:58.652 INFO:teuthology.orchestra.run.smithi170.stdout:2572 read 0x4bc8a00 thru 0x4bcb3b6 (0x29b7 bytes) 2024-09-21T23:15:58.680 INFO:teuthology.orchestra.run.smithi170.stdout:2574 write 0x5c05000 thru 0x5c106c9 (0xb6ca bytes) 2024-09-21T23:15:58.684 INFO:teuthology.orchestra.run.smithi170.stdout:2576 trunc from 0x5c10800 to 0x40cd800 2024-09-21T23:15:58.745 INFO:teuthology.orchestra.run.smithi170.stdout:2579 write 0x44e7200 thru 0x44eef38 (0x7d39 bytes) 2024-09-21T23:15:58.747 INFO:teuthology.orchestra.run.smithi170.stdout:2581 read 0x1b24a00 thru 0x1b30be8 (0xc1e9 bytes) 2024-09-21T23:15:58.752 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:58.747+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:58.758 INFO:teuthology.orchestra.run.smithi170.stdout:2582 trunc from 0x44ef000 to 0x79f9400 2024-09-21T23:15:58.780 INFO:teuthology.orchestra.run.smithi170.stdout:2584 write 0x668600 thru 0x66ce5c (0x485d bytes) 2024-09-21T23:15:58.783 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:15:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:15:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:58.789 INFO:teuthology.orchestra.run.smithi170.stdout:2586 trunc from 0x79f9400 to 0x5c50200 2024-09-21T23:15:58.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:15:58.811 INFO:teuthology.orchestra.run.smithi170.stdout:2587 write 0x2fc9000 thru 0x2fc9581 (0x582 bytes) 2024-09-21T23:15:58.847 INFO:teuthology.orchestra.run.smithi170.stdout:2594 write 0xa83f600 thru 0xa8490cd (0x9ace bytes) 2024-09-21T23:15:58.853 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:58.847+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:58.874 INFO:teuthology.orchestra.run.smithi170.stdout:2595 trunc from 0xa849200 to 0xd8d800 2024-09-21T23:15:58.935 INFO:teuthology.orchestra.run.smithi170.stdout:2599 trunc from 0xd8d800 to 0x4123e00 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.590883378999997, 2024-09-21T23:15:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:15:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.823241017999997, 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:15:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.443930936000001, 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:15:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:15:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.316713923999998, 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:15:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:15:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.401950192999998, 2024-09-21T23:15:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:15:58.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:15:58.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:58.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.946997426999999, 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:58.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:15:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.096971044, 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:15:58.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.096929625999998, 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:15:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.914843699999999, 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:15:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:15:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.772901357000002, 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:15:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.617035061999999, 2024-09-21T23:15:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:15:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:15:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.454304784999998, 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:15:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:15:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.934879402, 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:15:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.934750926, 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:15:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:15:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.933789222000001, 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:15:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.932778011, 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:15:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.931696064, 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:15:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:15:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.055117805999998, 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:15:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.054778236000001, 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:15:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:596 3.12 3:4d026f6e:::rbd_data.10f1dc9013.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.985063225000001, 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078799430000000004, 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:15:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 596 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:15:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951020+0000", 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-06 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951929+0000", 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090880699999999997 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951965+0000", 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5734e-05 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951982+0000", 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6806000000000001e-05 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952135+0000", 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015325899999999999 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952560+0000", 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042476500000000002 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.953181+0000", 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062147299999999999 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.957005+0000", 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038238 2024-09-21T23:15:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958342+0000", 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001336622 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958557+0000", 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021509599999999999 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958897+0000", 2024-09-21T23:15:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034014100000000002 2024-09-21T23:15:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:15:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:15:59.050 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:59.051 INFO:teuthology.orchestra.run.smithi170.stdout:2601 write 0x5b19600 thru 0x5b24c90 (0xb691 bytes) 2024-09-21T23:15:59.052 INFO:teuthology.orchestra.run.smithi170.stdout:2603 trunc from 0x5b24e00 to 0x49c5400 2024-09-21T23:15:59.052 INFO:teuthology.orchestra.run.smithi170.stdout:2604 read 0xa3dc00 thru 0xa43ebc (0x62bd bytes) 2024-09-21T23:15:59.052 INFO:teuthology.orchestra.run.smithi170.stdout:2607 read 0x2ba7e00 thru 0x2bafa55 (0x7c56 bytes) 2024-09-21T23:15:59.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:15:59.053 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:59.047+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:59.070 INFO:teuthology.orchestra.run.smithi170.stdout:2608 write 0x652ce00 thru 0x653b80f (0xea10 bytes) 2024-09-21T23:15:59.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:15:59.108 INFO:teuthology.orchestra.run.smithi170.stdout:2610 write 0x95da000 thru 0x95e05e0 (0x65e1 bytes) 2024-09-21T23:15:59.109 INFO:teuthology.orchestra.run.smithi170.stdout:2611 read 0x3eecc00 thru 0x3efb531 (0xe932 bytes) 2024-09-21T23:15:59.109 INFO:teuthology.orchestra.run.smithi170.stdout:2612 write 0x6909200 thru 0x6913d30 (0xab31 bytes) 2024-09-21T23:15:59.109 INFO:teuthology.orchestra.run.smithi170.stdout:2614 write 0x6329e00 thru 0x632e9a5 (0x4ba6 bytes) 2024-09-21T23:15:59.110 INFO:teuthology.orchestra.run.smithi170.stdout:2616 write 0x930f400 thru 0x931c816 (0xd417 bytes) 2024-09-21T23:15:59.110 INFO:teuthology.orchestra.run.smithi170.stdout:2617 read 0x1f8ee00 thru 0x1f9c0ed (0xd2ee bytes) 2024-09-21T23:15:59.110 INFO:teuthology.orchestra.run.smithi170.stdout:2618 write 0x572ce00 thru 0x573399e (0x6b9f bytes) 2024-09-21T23:15:59.110 INFO:teuthology.orchestra.run.smithi170.stdout:2619 read 0x1844800 thru 0x1848f31 (0x4732 bytes) 2024-09-21T23:15:59.110 INFO:teuthology.orchestra.run.smithi170.stdout:2621 read 0x3c3ee00 thru 0x3c4c5ae (0xd7af bytes) 2024-09-21T23:15:59.111 INFO:teuthology.orchestra.run.smithi170.stdout:2624 read 0x7a13200 thru 0x7a17760 (0x4561 bytes) 2024-09-21T23:15:59.111 INFO:teuthology.orchestra.run.smithi170.stdout:2625 read 0x5192000 thru 0x519f85f (0xd860 bytes) 2024-09-21T23:15:59.111 INFO:teuthology.orchestra.run.smithi170.stdout:2627 read 0x609600 thru 0x6181b3 (0xebb4 bytes) 2024-09-21T23:15:59.153 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:59.147+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:59.156 INFO:teuthology.orchestra.run.smithi170.stdout:2629 write 0xa628000 thru 0xa6280fb (0xfc bytes) 2024-09-21T23:15:59.159 INFO:teuthology.orchestra.run.smithi170.stdout:2630 read 0x2763400 thru 0x276c23b (0x8e3c bytes) 2024-09-21T23:15:59.159 INFO:teuthology.orchestra.run.smithi170.stdout:2631 write 0x294a600 thru 0x2955b01 (0xb502 bytes) 2024-09-21T23:15:59.167 INFO:teuthology.orchestra.run.smithi170.stdout:2633 trunc from 0xa628200 to 0xd057200 2024-09-21T23:15:59.195 INFO:teuthology.orchestra.run.smithi170.stdout:2634 punch from 0x6876400 to 0x6884a00, (0xe600 bytes) 2024-09-21T23:15:59.196 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:15:59.197 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:15:59.197 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:15:59.197 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:15:59.198 INFO:teuthology.orchestra.run.smithi170.stdout:2635 clone 14 order 23 su 1048576 sc 7 2024-09-21T23:15:59.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:15:59.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:15:59.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:15:59.237 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:15:59.253 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:59.247+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:15:59.259 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:15:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:15:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:15:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:15:59.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:15:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:15:59.319 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:15:59.340 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:15:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:15:59.344 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:15:59.344 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:15:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:15:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:15:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:15:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:15:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:15:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:15:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:config diff 2024-09-21T23:15:59.349 INFO:teuthology.orchestra.run.smithi053.stderr:config diff get 2024-09-21T23:15:59.349 INFO:teuthology.orchestra.run.smithi053.stderr:config get 2024-09-21T23:15:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:config help [] 2024-09-21T23:15:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:config set ... 2024-09-21T23:15:59.350 INFO:teuthology.orchestra.run.smithi053.stderr:admin_socket: invalid command 2024-09-21T23:15:59.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:15:59.351 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-21T23:15:59.356 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:59.356 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:59.356 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:59.356+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:59.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:15:59.464 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:59.464+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:15:59.535 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:59.536 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:59.550 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:15:59.592 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:15:59.592+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:15:59.657 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:59.651+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:59.684 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:59.684 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:59.698 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:15:59.783 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:15:59.779+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:15:59.858 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:15:59.851+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:15:59.861 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:15:59.862 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:15:59.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:15:59.959 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:15:59.956+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:00.060 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:00.055+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:00.078 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:00.078 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:00.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:00.136 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:16:00.186 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:00.183+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:00.261 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:00.255+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:00.272 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:00.272 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:00.277 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone13 from 0xd057200 (overlap 0x463200) to 0xe0a00 2024-09-21T23:16:00.287 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:00.378 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:00.375+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:00.457 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:00.457 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:00.463 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:00.459+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:00.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:00.564 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:00.564+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:00.637 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:00.637 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:00.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:00.699 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:00.695+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:00.766 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:00.764+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:00.775 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:00.775 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:00.790 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:00.810 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:00.808+0000 7f9740dfd7c0 -1 osd.0 29 log_to_monitors true 2024-09-21T23:16:00.853 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-21T23:16:00.867 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:00.864+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:00.961 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:00.961 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:00.968 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:00.963+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:00.978 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:01.070 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:01.067+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:01.170 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:01.170 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:01.170 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:01.168+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:01.176 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:01.176+0000 7f9734bca640 -1 osd.0 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T23:16:01.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:01.271 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:01.267+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:01.328 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:01.329 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:01.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:01.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:01.393 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:01.392+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:01.472 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:01.472+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:01.481 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:01.481 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:01.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:01.520 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T23:16:01.539 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:01.539 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:01.557 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:01.557 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:01.573 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:01.572+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:01.580 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:01.603 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:01.604 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:01.625 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:01.628 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T23:16:01.629 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T23:16:01.639 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:01.639 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:01.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:01.639 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.294867232000001, 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:16:01.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:01.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:16:01.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.527224871000001, 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:16:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:16:01.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.147914788999998, 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:01.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:16:01.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.020697777000002, 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:16:01.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:01.666 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.105934046000002, 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:16:01.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:16:01.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:16:01.673 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.650981280000003, 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:16:01.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:16:01.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.800954896999997, 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:16:01.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:16:01.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.800913479000002, 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.692 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.618827553000003, 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:16:01.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:16:01.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:01.699 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.476885209999999, 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.321018915000003, 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:01.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:01.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:16:01.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:16:01.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:16:01.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.158288638000002, 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:16:01.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.638863254999997, 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:16:01.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:16:01.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.638734779000004, 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:16:01.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.637773074999998, 2024-09-21T23:16:01.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:16:01.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:16:01.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.636761864, 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:16:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:16:01.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:16:01.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:16:01.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.635679916999997, 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:16:01.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:16:01.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.759101658999999, 2024-09-21T23:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:16:01.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:16:01.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.758762089000001, 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:16:01.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:596 3.12 3:4d026f6e:::rbd_data.10f1dc9013.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.689047078000002, 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078799430000000004, 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:01.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 596 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:01.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951020+0000", 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-06 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951929+0000", 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090880699999999997 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951965+0000", 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5734e-05 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951982+0000", 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6806000000000001e-05 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952135+0000", 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015325899999999999 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:01.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952560+0000", 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042476500000000002 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.953181+0000", 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062147299999999999 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.957005+0000", 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038238 2024-09-21T23:16:01.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958342+0000", 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001336622 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958557+0000", 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021509599999999999 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:01.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958897+0000", 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034014100000000002 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:01.763 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:01.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:01.765 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:01.759+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:01.865 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:01.864+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:01.957 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:01.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:01.958 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:01.958 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:01.966 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:01.964+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:01.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:02.067 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:02.063+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:02.150 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:02.150 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:02.151 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:02.151 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:02.151 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:02.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:16:02.167 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:02.164+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:02.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:02.268 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:02.263+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:02.346 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:02.346 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.417636521999999, 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:02.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:02.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416615674999999, 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:02.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416556002, 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:02.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:02.362 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416466025999998, 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:02.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416416116000001, 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:02.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:02.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416359975999999, 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:02.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416306193, 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:02.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416273917000002, 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:02.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:02.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:02.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416243637000001, 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:02.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416213980999999, 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:02.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416184013999999, 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:02.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416106185, 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:02.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:02.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416058478, 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:02.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.416028281999999, 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:02.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:02.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.415998365, 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:02.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.415968565, 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:02.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:02.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.415938632, 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:02.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:02.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.415873669, 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:02.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:02.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.415843193000001, 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:02.444 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.415814029, 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:02.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:02.449 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:02.451 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:02.451 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:02.448+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:02.552 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:02.547+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:02.582 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:02.582 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:02.582 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:02.582 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:02.596 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:02.687 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:02.683+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:02.754 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:02.751+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:02.762 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:02.762 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:02.762 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:02.762 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:02.762 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:02.776 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:02.855 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:02.852+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:02.942 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.719262345000004, 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.943 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:16:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:16:02.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:16:02.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.206857061999997, 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:16:02.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:16:02.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.206792159000003, 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.956 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:16:02.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:16:02.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.206645719999997, 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:16:02.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:16:02.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.206576604999995, 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.968 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:16:02.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.205964280000003, 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:16:02.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:16:02.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.979 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.205759874999998, 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.980 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:16:02.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:16:02.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:16:02.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.985 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.205717089000004, 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.986 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:16:02.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:16:02.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:16:02.993 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.993 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.993 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814758800999996, 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:16:02.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:02.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:16:02.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:02.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814672290999994, 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.000 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:16:03.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814595401999995, 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.006 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:16:03.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:16:03.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.011 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814390562, 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:16:03.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814302280000007, 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:16:03.017 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:16:03.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:16:03.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:16:03.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814171451999997, 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:03.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:16:03.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:16:03.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814125798999996, 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:16:03.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.034 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.814087372000003, 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:16:03.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:16:03.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.040 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.813957909999999, 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:16:03.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:16:03.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:16:03.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.813936268000006, 2024-09-21T23:16:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:16:03.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:16:03.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.813916223999996, 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.052 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:16:03.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:16:03.057 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.813832231999996, 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:03.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:16:03.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.063 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:03.065 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:03.065 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:03.059+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:03.152 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:16:03.166 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:03.159+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4900.0:8094 3.b 3.9143096b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:02.150237+0000", 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 1.08832833, 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08835456, 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.240 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4900", 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 8094 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150237+0000", 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150237+0000", 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150238+0000", 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.167e-06 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150245+0000", 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6899999999999992e-06 2024-09-21T23:16:03.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150267+0000", 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.19e-05 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150282+0000", 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4856e-05 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150405+0000", 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000123106 2024-09-21T23:16:03.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.150421+0000", 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6133e-05 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.164141+0000", 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013719373999999999 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:02.164168+0000", 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6990000000000001e-05 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.163538+0000", 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.99937045899999999 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.163557+0000", 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8533999999999999e-05 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:03.164097+0000", 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.074467700999999997, 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.074574477, 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4900", 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 8098 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164097+0000", 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164097+0000", 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164099+0000", 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.95e-06 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164106+0000", 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8600000000000004e-06 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164122+0000", 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6229e-05 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164137+0000", 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4781e-05 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164437+0000", 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030002799999999998 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164448+0000", 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1381e-05 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4900.0:8099 3.b 3.9143096b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:03.164187+0000", 2024-09-21T23:16:03.249 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.074378200000000005, 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.074536755999999996, 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4900", 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 8099 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164187+0000", 2024-09-21T23:16:03.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164187+0000", 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164188+0000", 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1100000000000004e-07 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164198+0000", 2024-09-21T23:16:03.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0329000000000001e-05 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164211+0000", 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2771e-05 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164220+0000", 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0739999999999999e-06 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164315+0000", 2024-09-21T23:16:03.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5451999999999993e-05 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:03.164324+0000", 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2819999999999996e-06 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 3 2024-09-21T23:16:03.253 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:03.255 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:03.266 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:03.264+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:03.368 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:03.363+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:03.424 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:03.424 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:03.424 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:03.424 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:03.424 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:03.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:03.483 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:03.479+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:03.569 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:03.563+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.650777210000001, 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.609 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:16:03.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.650510226000002, 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:16:03.615 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:16:03.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:16:03.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:16:03.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3357 3.11 3:8fb7664f:::rbd_data.10f1cfc9677.0000000000000015:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.431556383, 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049070639999999999, 2024-09-21T23:16:03.621 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3357 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173029+0000", 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.494e-06 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173037+0000", 2024-09-21T23:16:03.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173060+0000", 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717999999999998e-05 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173074+0000", 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4671000000000001e-05 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204211+0000", 2024-09-21T23:16:03.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031136891999999999 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204347+0000", 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013595400000000001 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222098+0000", 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017750717999999999 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3418 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423537619000001, 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054653000000003, 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3418 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181047+0000", 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6899999999999996e-07 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181057+0000", 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0223e-05 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181064+0000", 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9949999999999999e-06 2024-09-21T23:16:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181069+0000", 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4160000000000003e-06 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221944+0000", 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040874684000000001 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222101+0000", 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015656599999999999 2024-09-21T23:16:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3419 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423506625000002, 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041067436999999998, 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3419 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181080+0000", 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.402e-06 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181087+0000", 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181092+0000", 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5720000000000002e-06 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222136+0000", 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041044316999999997 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222144+0000", 2024-09-21T23:16:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9349999999999994e-06 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3420 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423484202000001, 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041092287999999998, 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3420 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181100+0000", 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8500000000000001e-07 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181102+0000", 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2879999999999999e-06 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181109+0000", 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181115+0000", 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7819999999999999e-06 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222181+0000", 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041065989999999997 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222192+0000", 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.076e-05 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3421 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423461704000001, 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041102946000000001, 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3421 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181125+0000", 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5390000000000002e-06 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181132+0000", 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6610000000000003e-06 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181137+0000", 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2429999999999996e-06 2024-09-21T23:16:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222217+0000", 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041080113000000001 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222225+0000", 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9049999999999997e-06 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3422 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423439123000001, 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041150084000000003, 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3422 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181145+0000", 2024-09-21T23:16:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181148+0000", 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.57e-06 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181154+0000", 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9909999999999999e-06 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181160+0000", 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222287+0000", 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041126708999999997 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222295+0000", 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6799999999999993e-06 2024-09-21T23:16:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3424 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423416251999999, 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041160906999999997, 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3424 2024-09-21T23:16:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181168+0000", 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181170+0000", 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181178+0000", 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181183+0000", 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3410000000000002e-06 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222321+0000", 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041137765 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222328+0000", 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6550000000000004e-06 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3425 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423393397000002, 2024-09-21T23:16:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041445086999999999, 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3425 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181191+0000", 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999999e-07 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181193+0000", 2024-09-21T23:16:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6800000000000002e-06 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181200+0000", 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6760000000000001e-06 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181205+0000", 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.383e-06 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222627+0000", 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041421342 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222635+0000", 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6130000000000004e-06 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3509 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.423012752999998, 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040950634, 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3509 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181574+0000", 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.199e-06 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181580+0000", 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0560000000000001e-06 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181585+0000", 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1669999999999998e-06 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222516+0000", 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931568000000002 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222521+0000", 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3510 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.422990012, 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040943715999999998, 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3510 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.32e-07 2024-09-21T23:16:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181596+0000", 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.339e-06 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181603+0000", 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181608+0000", 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222534+0000", 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040925221999999997 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222537+0000", 2024-09-21T23:16:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3567 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.422626877999999, 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040956498000000001, 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3567 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7899999999999999e-07 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181959+0000", 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1830000000000001e-06 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181966+0000", 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3119999999999999e-06 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181973+0000", 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222902+0000", 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928961999999999 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222913+0000", 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1552000000000001e-05 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:499 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.824672230999999, 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041124135999999999, 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 499 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779913+0000", 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779918+0000", 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779938+0000", 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9553000000000001e-05 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779957+0000", 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820711+0000", 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040754603 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820848+0000", 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013633699999999999 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821036+0000", 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018796899999999999 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:500 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.824607091000001, 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041461570000000003, 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 500 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779978+0000", 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1969999999999999e-06 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779983+0000", 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780045+0000", 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2082000000000006e-05 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780087+0000", 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1863000000000002e-05 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821117+0000", 2024-09-21T23:16:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041030076999999998 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821247+0000", 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012996199999999999 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821438+0000", 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019079899999999999 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:501 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.824413358999999, 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042023224999999997, 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 501 2024-09-21T23:16:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780172+0000", 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:16:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780184+0000", 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1039999999999999e-05 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780215+0000", 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1006000000000001e-05 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780235+0000", 2024-09-21T23:16:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0483999999999999e-05 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821553+0000", 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041318311000000003 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821848+0000", 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029481199999999998 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822193+0000", 2024-09-21T23:16:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034536499999999998 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:506 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.824314806, 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041920526, 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 506 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780270+0000", 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3370000000000001e-06 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780278+0000", 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8730000000000007e-06 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780300+0000", 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2183000000000001e-05 2024-09-21T23:16:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780316+0000", 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5413e-05 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821521+0000", 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041205506000000003 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821802+0000", 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028134500000000001 2024-09-21T23:16:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822189+0000", 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038686900000000003 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:507 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.82424494, 2024-09-21T23:16:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042296304, 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 507 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780340+0000", 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.31e-06 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780347+0000", 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780367+0000", 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9466e-05 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780382+0000", 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5274e-05 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822269+0000", 2024-09-21T23:16:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041886592 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822418+0000", 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014924299999999999 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822635+0000", 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021715499999999999 2024-09-21T23:16:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:508 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.824104487, 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042742988000000003, 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 508 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780481+0000", 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4869999999999999e-06 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780489+0000", 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1200000000000002e-06 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781280+0000", 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000791583 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781307+0000", 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6486999999999999e-05 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822728+0000", 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041420917000000002 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822937+0000", 2024-09-21T23:16:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000209593 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823222+0000", 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000284801 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:509 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.822859021999999, 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041990254999999997, 2024-09-21T23:16:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 509 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781726+0000", 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2759999999999999e-06 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781744+0000", 2024-09-21T23:16:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8377e-05 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781767+0000", 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3006999999999998e-05 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781780+0000", 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2393e-05 2024-09-21T23:16:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823275+0000", 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041494879999999998 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823485+0000", 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021036300000000001 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823715+0000", 2024-09-21T23:16:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022995900000000001 2024-09-21T23:16:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:03.702 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:03.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:03.704 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:03.705 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:03.699+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:03.804 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:03.799+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:03.860 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-21T23:16:03.876 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:03.876 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:03.878 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:03.879 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:03.879 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:03.879 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:03.893 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:03.894 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:03.894 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:03.912 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:03.912 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:03.931 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:03.938 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:03.935+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:03.951 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:03.951 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:04.006 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:04.003+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:04.060 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:04.060 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:04.074 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:04.107 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:04.104+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:04.208 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:04.208+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:04.249 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:04.249 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:04.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:04.312 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:04.312+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:04.351 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:04.397 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:04.397 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:04.410 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:04.408+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:04.413 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:04.491 INFO:teuthology.orchestra.run.smithi170.stdout:2636 trunc from 0xd057200 to 0x3dd800 2024-09-21T23:16:04.501 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:04.501 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:04.512 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:04.507+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:04.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:16:04.569 INFO:teuthology.orchestra.run.smithi170.stdout:2637 trunc from 0x3dd800 to 0xa868200 2024-09-21T23:16:04.589 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:04.589 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:04.593 INFO:teuthology.orchestra.run.smithi170.stdout:2641 write 0x4151e00 thru 0x41529f9 (0xbfa bytes) 2024-09-21T23:16:04.595 INFO:teuthology.orchestra.run.smithi170.stdout:2642 read 0x94b5c00 thru 0x94bdedc (0x82dd bytes) 2024-09-21T23:16:04.595 INFO:teuthology.orchestra.run.smithi170.stdout:2649 read 0xbc7c00 thru 0xbcba5d (0x3e5e bytes) 2024-09-21T23:16:04.595 INFO:teuthology.orchestra.run.smithi170.stdout:2652 read 0x266e400 thru 0x2670fdd (0x2bde bytes) 2024-09-21T23:16:04.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:04.629 INFO:teuthology.orchestra.run.smithi170.stdout:2655 write 0xb862000 thru 0xb86a1d1 (0x81d2 bytes) 2024-09-21T23:16:04.646 INFO:teuthology.orchestra.run.smithi170.stdout:2659 trunc from 0xb86a200 to 0x1af5a00 2024-09-21T23:16:04.656 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:04.656+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:04.694 INFO:teuthology.orchestra.run.smithi170.stdout:2661 read 0xd63600 thru 0xd6d6b6 (0xa0b7 bytes) 2024-09-21T23:16:04.713 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:04.712+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:04.717 INFO:teuthology.orchestra.run.smithi170.stdout:2663 trunc from 0x1af5a00 to 0xae7a600 2024-09-21T23:16:04.743 INFO:teuthology.orchestra.run.smithi170.stdout:2664 punch from 0x9ed3c00 to 0x9edaa00, (0x6e00 bytes) 2024-09-21T23:16:04.744 INFO:teuthology.orchestra.run.smithi170.stdout:2665 read 0x6f38400 thru 0x6f3eca6 (0x68a7 bytes) 2024-09-21T23:16:04.745 INFO:teuthology.orchestra.run.smithi170.stdout:2667 read 0x78b7800 thru 0x78c2f6b (0xb76c bytes) 2024-09-21T23:16:04.745 INFO:teuthology.orchestra.run.smithi170.stdout:2668 punch from 0x3589400 to 0x3594600, (0xb200 bytes) 2024-09-21T23:16:04.746 INFO:teuthology.orchestra.run.smithi170.stdout:2669 read 0x2abb000 thru 0x2ac6e30 (0xbe31 bytes) 2024-09-21T23:16:04.747 INFO:teuthology.orchestra.run.smithi170.stdout:2671 punch from 0x3e2800 to 0x3e7600, (0x4e00 bytes) 2024-09-21T23:16:04.747 INFO:teuthology.orchestra.run.smithi170.stdout:2672 punch from 0x9c86200 to 0x9c94600, (0xe400 bytes) 2024-09-21T23:16:04.747 INFO:teuthology.orchestra.run.smithi170.stdout:2673 clone 15 order 19 su 2048 sc 8 2024-09-21T23:16:04.801 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:04.801 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:04.814 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:04.812+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:04.817 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:04.915 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:04.912+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:04.997 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:04.997 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:05.012 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:05.060 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:05.060+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:05.117 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:05.116+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:05.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:16:05.198 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:05.198 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:05.199 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:16:05.217 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:05.216+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:05.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:05.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:16:05.324 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:05.324+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:05.397 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:05.397 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:05.417 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:05.420 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:05.415+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:05.519 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:05.515+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:05.548 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:05.548 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:05.563 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:05.651 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:05.647+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:05.721 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:05.715+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:05.730 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:05.730 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:05.745 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:05.831 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:05.827+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:05.904 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:05.905 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:05.919 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:05.963 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:05.959+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:05.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:06.024 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:06.024+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:06.087 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:06.087 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:06.101 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:06.124 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:06.124+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:06.130 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:06.147 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:06.147 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:06.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:06.163 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:06.182 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:06.182 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:06.199 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:16:06.200 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:16:06.216 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:06.216 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:06.225 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:06.224+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:06.230 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:16:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:06.232 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:06.232 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:06.232 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:06.237 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:06.237 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:06.246 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:06.326 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:06.324+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:06.371 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.414736447999999, 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:16:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.414469464, 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:16:06.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3357 3.11 3:8fb7664f:::rbd_data.10f1cfc9677.0000000000000015:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.195515620999998, 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049070639999999999, 2024-09-21T23:16:06.383 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3357 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173029+0000", 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.494e-06 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173037+0000", 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173060+0000", 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717999999999998e-05 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173074+0000", 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4671000000000001e-05 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204211+0000", 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031136891999999999 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204347+0000", 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013595400000000001 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222098+0000", 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017750717999999999 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3418 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.187496856999999, 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054653000000003, 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3418 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181047+0000", 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6899999999999996e-07 2024-09-21T23:16:06.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181057+0000", 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0223e-05 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181064+0000", 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9949999999999999e-06 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181069+0000", 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4160000000000003e-06 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221944+0000", 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040874684000000001 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222101+0000", 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015656599999999999 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.391 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3419 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.187465863, 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041067436999999998, 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3419 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181080+0000", 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.402e-06 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181087+0000", 2024-09-21T23:16:06.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181092+0000", 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5720000000000002e-06 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222136+0000", 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041044316999999997 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222144+0000", 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9349999999999994e-06 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3420 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.187443439999999, 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041092287999999998, 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.396 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3420 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:06.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181100+0000", 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8500000000000001e-07 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181102+0000", 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2879999999999999e-06 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181109+0000", 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181115+0000", 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7819999999999999e-06 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222181+0000", 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041065989999999997 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222192+0000", 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.076e-05 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3421 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.187420941999999, 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041102946000000001, 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3421 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.401 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:06.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181125+0000", 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5390000000000002e-06 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181132+0000", 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6610000000000003e-06 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181137+0000", 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2429999999999996e-06 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222217+0000", 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041080113000000001 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222225+0000", 2024-09-21T23:16:06.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9049999999999997e-06 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3422 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.187398361, 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041150084000000003, 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3422 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181145+0000", 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:16:06.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181148+0000", 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.57e-06 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181154+0000", 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9909999999999999e-06 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181160+0000", 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222287+0000", 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041126708999999997 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222295+0000", 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6799999999999993e-06 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3424 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.187375490000001, 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041160906999999997, 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.410 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3424 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181168+0000", 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181170+0000", 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181178+0000", 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181183+0000", 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3410000000000002e-06 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222321+0000", 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041137765 2024-09-21T23:16:06.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222328+0000", 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6550000000000004e-06 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3425 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.187352635, 2024-09-21T23:16:06.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041445086999999999, 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3425 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181191+0000", 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999999e-07 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181193+0000", 2024-09-21T23:16:06.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6800000000000002e-06 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181200+0000", 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6760000000000001e-06 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181205+0000", 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.383e-06 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222627+0000", 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041421342 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222635+0000", 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6130000000000004e-06 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3509 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.186971991, 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040950634, 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3509 2024-09-21T23:16:06.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181574+0000", 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.199e-06 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181580+0000", 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0560000000000001e-06 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181585+0000", 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1669999999999998e-06 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222516+0000", 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931568000000002 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222521+0000", 2024-09-21T23:16:06.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3510 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.186949250000001, 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040943715999999998, 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3510 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:06.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.32e-07 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181596+0000", 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.339e-06 2024-09-21T23:16:06.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181603+0000", 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181608+0000", 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222534+0000", 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040925221999999997 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222537+0000", 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3567 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.186586116000001, 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040956498000000001, 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3567 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7899999999999999e-07 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181959+0000", 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1830000000000001e-06 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181966+0000", 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3119999999999999e-06 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181973+0000", 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222902+0000", 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928961999999999 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222913+0000", 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1552000000000001e-05 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:499 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.588631468999999, 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041124135999999999, 2024-09-21T23:16:06.432 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 499 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779913+0000", 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779918+0000", 2024-09-21T23:16:06.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779938+0000", 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9553000000000001e-05 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779957+0000", 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:16:06.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820711+0000", 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040754603 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820848+0000", 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013633699999999999 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821036+0000", 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018796899999999999 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:500 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.588566329000001, 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041461570000000003, 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.437 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 500 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779978+0000", 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1969999999999999e-06 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779983+0000", 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780045+0000", 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2082000000000006e-05 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780087+0000", 2024-09-21T23:16:06.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1863000000000002e-05 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821117+0000", 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041030076999999998 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821247+0000", 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012996199999999999 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821438+0000", 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019079899999999999 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:501 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.588372596999999, 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042023224999999997, 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.442 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 501 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:06.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780172+0000", 2024-09-21T23:16:06.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:16:06.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780184+0000", 2024-09-21T23:16:06.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1039999999999999e-05 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780215+0000", 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1006000000000001e-05 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780235+0000", 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0483999999999999e-05 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821553+0000", 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041318311000000003 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821848+0000", 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029481199999999998 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822193+0000", 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034536499999999998 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.447 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:506 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.588274044, 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041920526, 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:06.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 506 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780270+0000", 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3370000000000001e-06 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780278+0000", 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8730000000000007e-06 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780300+0000", 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2183000000000001e-05 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780316+0000", 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5413e-05 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821521+0000", 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041205506000000003 2024-09-21T23:16:06.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821802+0000", 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028134500000000001 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822189+0000", 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038686900000000003 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:507 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.588204178, 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042296304, 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 507 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780340+0000", 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.31e-06 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780347+0000", 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780367+0000", 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9466e-05 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780382+0000", 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5274e-05 2024-09-21T23:16:06.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822269+0000", 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041886592 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822418+0000", 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014924299999999999 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822635+0000", 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021715499999999999 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:508 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.588063725, 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042742988000000003, 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.457 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 508 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780481+0000", 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4869999999999999e-06 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780489+0000", 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1200000000000002e-06 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781280+0000", 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000791583 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781307+0000", 2024-09-21T23:16:06.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6486999999999999e-05 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822728+0000", 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041420917000000002 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822937+0000", 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000209593 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823222+0000", 2024-09-21T23:16:06.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000284801 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:509 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.586818259999999, 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041990254999999997, 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 509 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781726+0000", 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2759999999999999e-06 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781744+0000", 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8377e-05 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:06.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781767+0000", 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3006999999999998e-05 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781780+0000", 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2393e-05 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823275+0000", 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041494879999999998 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823485+0000", 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021036300000000001 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823715+0000", 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022995900000000001 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:06.466 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:06.468 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:06.468 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:06.463+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:06.568 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:06.563+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:06.637 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:06.638 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:06.638 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:06.638 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:06.653 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:06.670 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:06.668+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:06.770 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:06.767+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:06.817 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:06.817 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:06.817 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:06.817 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:06.817 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:06.831 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:06.875 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:06.871+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:06.972 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:06.972+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:07.001 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3198 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 2244608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.812067215999999, 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034480396000000003, 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3198 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185426+0000", 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185427+0000", 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8400000000000003e-07 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185430+0000", 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.275e-06 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185439+0000", 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6500000000000002e-06 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.185454+0000", 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4778e-05 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213252+0000", 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027798665 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.213316+0000", 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3701999999999994e-05 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219907+0000", 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065904420000000002 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3349 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2179072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.809028472000001, 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029976903999999999, 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3349 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188465+0000", 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:16:07.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188467+0000", 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1390000000000002e-06 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188473+0000", 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0159999999999997e-06 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188478+0000", 2024-09-21T23:16:07.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5480000000000003e-06 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218328+0000", 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029849969000000001 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218367+0000", 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9474999999999997e-05 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218442+0000", 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4418999999999993e-05 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3350 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2260992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.809004658999999, 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030043218999999999, 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3350 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:16:07.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188489+0000", 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188492+0000", 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2419999999999999e-06 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188499+0000", 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0670000000000003e-06 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188504+0000", 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9699999999999998e-06 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218457+0000", 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029953832999999999 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218493+0000", 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5216000000000003e-05 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218532+0000", 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9267000000000002e-05 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.015 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3351 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2342912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.808982583999999, 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030166889999999998, 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3351 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188511+0000", 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:16:07.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188514+0000", 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5210000000000001e-06 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188520+0000", 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4799999999999998e-06 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188526+0000", 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3179999999999997e-06 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218555+0000", 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029059 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218613+0000", 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8084999999999997e-05 2024-09-21T23:16:07.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218678+0000", 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5021e-05 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3352 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2424832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.808959065, 2024-09-21T23:16:07.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.03030147, 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3352 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:16:07.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188534+0000", 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188535+0000", 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188540+0000", 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6790000000000002e-06 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188547+0000", 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2439999999999998e-06 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188551+0000", 2024-09-21T23:16:07.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7310000000000001e-06 2024-09-21T23:16:07.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218703+0000", 2024-09-21T23:16:07.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030151305999999999 2024-09-21T23:16:07.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218768+0000", 2024-09-21T23:16:07.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5512999999999994e-05 2024-09-21T23:16:07.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218836+0000", 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7550000000000002e-05 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3353 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2506752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.808932738999999, 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030431269, 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3353 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.120 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188561+0000", 2024-09-21T23:16:07.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8599999999999998e-07 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188564+0000", 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4229999999999999e-06 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188570+0000", 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1199999999999999e-06 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188577+0000", 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.221e-06 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218860+0000", 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030282878999999999 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218924+0000", 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3948999999999994e-05 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.218992+0000", 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190999999999998e-05 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3354 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 2588672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.808910316999999, 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030565452, 2024-09-21T23:16:07.124 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3354 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188583+0000", 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188586+0000", 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1919999999999999e-06 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188592+0000", 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2380000000000002e-06 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.188597+0000", 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0819999999999998e-06 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219018+0000", 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030420823 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219084+0000", 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6513000000000005e-05 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219149+0000", 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4187000000000001e-05 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3460 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807943415, 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029906078999999999, 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.129 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3460 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189550+0000", 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189553+0000", 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3010000000000001e-06 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189559+0000", 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4819999999999999e-06 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189564+0000", 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1819999999999997e-06 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219323+0000", 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029758623000000001 2024-09-21T23:16:07.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219391+0000", 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8330000000000005e-05 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219456+0000", 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4747999999999997e-05 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3462 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807898299000001, 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030022725, 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3462 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189595+0000", 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189596+0000", 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:16:07.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189598+0000", 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8260000000000001e-06 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189605+0000", 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6919999999999996e-06 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.189611+0000", 2024-09-21T23:16:07.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2360000000000001e-06 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219480+0000", 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029868714000000001 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219547+0000", 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7268999999999996e-05 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219618+0000", 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0508999999999999e-05 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3469 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807425669000001, 2024-09-21T23:16:07.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030029238999999999, 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3469 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190068+0000", 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190069+0000", 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2500000000000004e-07 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190071+0000", 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9520000000000001e-06 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190075+0000", 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8420000000000004e-06 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190080+0000", 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4800000000000003e-06 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.219961+0000", 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029880803000000001 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220030+0000", 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8856999999999995e-05 2024-09-21T23:16:07.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220097+0000", 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7379999999999995e-05 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3470 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.80740741, 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030180927999999999, 2024-09-21T23:16:07.143 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3470 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:16:07.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190086+0000", 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190087+0000", 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5030000000000001e-06 2024-09-21T23:16:07.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190090+0000", 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.278e-06 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190095+0000", 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8910000000000003e-06 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190100+0000", 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9710000000000003e-06 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220128+0000", 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030028195000000001 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220198+0000", 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0405000000000001e-05 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220267+0000", 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8684999999999997e-05 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3472 3.1c 3:393e9721:::rbd_data.10f1dc9013.0000000000000009:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807367331999998, 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030301255999999999, 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3472 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190126+0000", 2024-09-21T23:16:07.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190127+0000", 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.238e-06 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190130+0000", 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5849999999999998e-06 2024-09-21T23:16:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190135+0000", 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0270000000000003e-06 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190140+0000", 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8609999999999997e-06 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220297+0000", 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030157498000000001 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220368+0000", 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0811999999999999e-05 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220427+0000", 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9234999999999998e-05 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3484 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3538944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807289145999999, 2024-09-21T23:16:07.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030347946000000001, 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3484 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190204+0000", 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190205+0000", 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.13e-06 2024-09-21T23:16:07.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190222+0000", 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6909999999999999e-05 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190229+0000", 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2979999999999997e-06 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190234+0000", 2024-09-21T23:16:07.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7130000000000002e-06 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220443+0000", 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030209037000000001 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220490+0000", 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6198000000000001e-05 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220552+0000", 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.266e-05 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3486 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3620864~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807231546000001, 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030430263999999999, 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3486 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:16:07.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190262+0000", 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190264+0000", 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.007e-06 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190270+0000", 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190275+0000", 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8670000000000002e-06 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220569+0000", 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030294237000000002 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220625+0000", 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5902000000000001e-05 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220692+0000", 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6796000000000002e-05 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.163 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3487 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3702784~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807211371000001, 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030517598, 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3487 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.164 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190282+0000", 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190284+0000", 2024-09-21T23:16:07.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.66e-06 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190287+0000", 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9359999999999999e-06 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190292+0000", 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5219999999999998e-06 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190297+0000", 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0900000000000004e-06 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220721+0000", 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030423823999999999 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220759+0000", 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7962e-05 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220800+0000", 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0604e-05 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3489 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3784704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807170427999999, 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030596542000000001, 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.168 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3489 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190323+0000", 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190324+0000", 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190327+0000", 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6180000000000002e-06 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190333+0000", 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7259999999999999e-06 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190338+0000", 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1680000000000003e-06 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220825+0000", 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030487104000000001 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220863+0000", 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8563000000000003e-05 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220920+0000", 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6051999999999998e-05 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3490 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3866624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807148700999999, 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030740849000000001, 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3490 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190345+0000", 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190346+0000", 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5859999999999999e-06 2024-09-21T23:16:07.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190349+0000", 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190354+0000", 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1950000000000002e-06 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190359+0000", 2024-09-21T23:16:07.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9819999999999999e-06 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.220944+0000", 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030585309000000001 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221006+0000", 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1802000000000002e-05 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221086+0000", 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9314999999999995e-05 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3491 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 3948544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807113336, 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030871663000000001, 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3491 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190380+0000", 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190382+0000", 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.522e-06 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190384+0000", 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8650000000000001e-06 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190391+0000", 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2899999999999999e-06 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190396+0000", 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9570000000000001e-06 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221112+0000", 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030716753999999999 2024-09-21T23:16:07.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221179+0000", 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6122000000000002e-05 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221252+0000", 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3152999999999996e-05 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3493 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807069652999999, 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030983984999999999, 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3493 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190424+0000", 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2800000000000002e-07 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190427+0000", 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4660000000000002e-06 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190433+0000", 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3110000000000002e-06 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190438+0000", 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7539999999999997e-06 2024-09-21T23:16:07.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221282+0000", 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030844399000000002 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221345+0000", 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3145999999999995e-05 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221408+0000", 2024-09-21T23:16:07.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2481e-05 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3495 3.1c 3:3f21163a:::rbd_data.10f1dc9013.000000000000000b:9 [read 4112384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.807026911000001, 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031095845, 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:07.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3495 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190466+0000", 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190468+0000", 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2219999999999999e-06 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190470+0000", 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5509999999999998e-06 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190475+0000", 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9599999999999999e-06 2024-09-21T23:16:07.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.190481+0000", 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4389999999999999e-06 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221436+0000", 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030955580999999999 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221501+0000", 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5030999999999995e-05 2024-09-21T23:16:07.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221562+0000", 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1061000000000004e-05 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:07.190 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:07.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:07.192 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:07.187+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:07.205 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:16:07.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:16:07.300 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone14 from 0xae7a600 (overlap 0x3dd800) to 0x18bc00 2024-09-21T23:16:07.321 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:07.320+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:07.325 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:07.325 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:07.325 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:07.325 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:07.342 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:07.393 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:07.387+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:07.494 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:07.492+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:07.518 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:07.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:07.519 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:07.519 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:07.519 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:07.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:07.595 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:07.591+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:07.696 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:07.691+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.5484113449999999, 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:16:07.711 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:16:07.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.4831565870000001, 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:16:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:661 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:head [delete] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.314955+0000", 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.39484058399999999, 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003392478, 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 661 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.314955+0000", 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.314955+0000", 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.314957+0000", 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3489999999999999e-06 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.314975+0000", 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7499e-05 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.314991+0000", 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6588000000000001e-05 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315007+0000", 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5526999999999999e-05 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315119+0000", 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111943 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315291+0000", 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017221700000000001 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315498+0000", 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000206691 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.316707+0000", 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001209187 2024-09-21T23:16:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.318167+0000", 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014600209999999999 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.318252+0000", 2024-09-21T23:16:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4952999999999998e-05 2024-09-21T23:16:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.318347+0000", 2024-09-21T23:16:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5502999999999997e-05 2024-09-21T23:16:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:662 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:head [delete] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.315038+0000", 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.39475788299999998, 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034280140000000001, 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 662 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315038+0000", 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315038+0000", 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315038+0000", 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7100000000000002e-07 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315043+0000", 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3329999999999998e-06 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315057+0000", 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4761e-05 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315066+0000", 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1609999999999992e-06 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315128+0000", 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1180999999999996e-05 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315295+0000", 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016744700000000001 2024-09-21T23:16:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.315498+0000", 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020261699999999999 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.316527+0000", 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010289170000000001 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.318404+0000", 2024-09-21T23:16:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001877667 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.318434+0000", 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9920000000000002e-05 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.318466+0000", 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439000000000002e-05 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:671 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [stat,truncate 1048576] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.329215+0000", 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.380580792, 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022120909999999998, 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 671 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329215+0000", 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329215+0000", 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329217+0000", 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4600000000000002e-06 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329231+0000", 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3845999999999999e-05 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329269+0000", 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8046999999999997e-05 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329288+0000", 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8510000000000001e-05 2024-09-21T23:16:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329389+0000", 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329544+0000", 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155632 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329724+0000", 2024-09-21T23:16:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000179345 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331253+0000", 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001529229 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331312+0000", 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9438999999999998e-05 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331370+0000", 2024-09-21T23:16:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7976999999999997e-05 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331427+0000", 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.647e-05 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4894.0:672 3.18 e66/64 3:1c55e7b5:::rbd_data.10f1e9e481e5.0000000000000001:head v 66'323, mlcod=0'0)", 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.329839+0000", 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.37995655499999997, 2024-09-21T23:16:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098011600000000006, 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329839+0000", 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329839+0000", 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329840+0000", 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.325e-06 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329849+0000", 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1260000000000004e-06 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329865+0000", 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5608e-05 2024-09-21T23:16:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329875+0000", 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8060000000000006e-06 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329972+0000", 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7433e-05 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.329990+0000", 2024-09-21T23:16:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8161000000000001e-05 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330778+0000", 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078735899999999995 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330819+0000", 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1298000000000003e-05 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:673 3.1f 3:fc6113ea:::rbd_data.10f1e9e481e5.0000000000000002:head [stat,delete] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.330094+0000", 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.37970099699999998, 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015677200000000001, 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 673 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330094+0000", 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330094+0000", 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330096+0000", 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.243e-06 2024-09-21T23:16:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330103+0000", 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5100000000000001e-06 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330123+0000", 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.944e-05 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330134+0000", 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1447000000000001e-05 2024-09-21T23:16:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330205+0000", 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1191000000000003e-05 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330307+0000", 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101346 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.330471+0000", 2024-09-21T23:16:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000164069 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331293+0000", 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082186499999999997 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331591+0000", 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029880599999999998 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331622+0000", 2024-09-21T23:16:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0694999999999997e-05 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.331662+0000", 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0108000000000003e-05 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:677 3.6 3:60fb81db:::rbd_data.10f1e9e481e5.0000000000000003:head [stat,delete] snapc 11=[11] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.337951+0000", 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.371844544, 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003176449, 2024-09-21T23:16:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 677 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.337951+0000", 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.337951+0000", 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.337954+0000", 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.337964+0000", 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0702e-05 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.337992+0000", 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7508000000000002e-05 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.338005+0000", 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3199e-05 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.338089+0000", 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3846999999999999e-05 2024-09-21T23:16:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.338218+0000", 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129338 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.338386+0000", 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167422 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.339495+0000", 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001109369 2024-09-21T23:16:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.341010+0000", 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001515064 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.341075+0000", 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5061000000000006e-05 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.341127+0000", 2024-09-21T23:16:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2265000000000002e-05 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:702 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.374560+0000", 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33523541899999998, 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091626299999999995, 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 702 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374560+0000", 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374560+0000", 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374561+0000", 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374563+0000", 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.818e-06 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374571+0000", 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4900000000000003e-06 2024-09-21T23:16:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374576+0000", 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6330000000000001e-06 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375283+0000", 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070664499999999999 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375364+0000", 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0770000000000001e-05 2024-09-21T23:16:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375476+0000", 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011243600000000001 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:703 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.374584+0000", 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33521109199999999, 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010727600000000001, 2024-09-21T23:16:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 703 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374584+0000", 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374584+0000", 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374585+0000", 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374588+0000", 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8399999999999999e-06 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374596+0000", 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9489999999999996e-06 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374602+0000", 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7970000000000003e-06 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375518+0000", 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091526800000000001 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375568+0000", 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9954000000000003e-05 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375657+0000", 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9586999999999995e-05 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:704 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.374611+0000", 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33518454600000003, 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001292367, 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 704 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374611+0000", 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374611+0000", 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374611+0000", 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374620+0000", 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1370000000000002e-06 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374628+0000", 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7059999999999993e-06 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374634+0000", 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5949999999999998e-06 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375705+0000", 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010715989999999999 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375790+0000", 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4257000000000003e-05 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375903+0000", 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000113647 2024-09-21T23:16:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:705 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.374644+0000", 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33515160700000002, 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014859980000000001, 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 705 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374644+0000", 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374644+0000", 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374644+0000", 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374647+0000", 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.497e-06 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374654+0000", 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-21T23:16:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374660+0000", 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2669999999999997e-06 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.375936+0000", 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001276683 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376017+0000", 2024-09-21T23:16:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0467000000000001e-05 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376130+0000", 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011313499999999999 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:706 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.374667+0000", 2024-09-21T23:16:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.335128174, 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001635044, 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 706 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374667+0000", 2024-09-21T23:16:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374667+0000", 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374668+0000", 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374670+0000", 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5720000000000001e-06 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374687+0000", 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6461e-05 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.374693+0000", 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5740000000000001e-06 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376185+0000", 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001491502 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376239+0000", 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4601000000000001e-05 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376302+0000", 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2978999999999995e-05 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:711 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 6656~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.376933+0000", 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33286297100000001, 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086507100000000003, 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376933+0000", 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376933+0000", 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376933+0000", 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376939+0000", 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4559999999999997e-06 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376947+0000", 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4120000000000004e-06 2024-09-21T23:16:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376952+0000", 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4820000000000002e-06 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377603+0000", 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00065104799999999999 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377686+0000", 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2478000000000006e-05 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377798+0000", 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111778 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:712 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 6144~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.376960+0000", 2024-09-21T23:16:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33283503599999997, 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010750810000000001, 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 712 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376960+0000", 2024-09-21T23:16:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376960+0000", 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376961+0000", 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8799999999999998e-07 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376963+0000", 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1859999999999999e-06 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376970+0000", 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7900000000000002e-06 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376975+0000", 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0520000000000001e-06 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377839+0000", 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086368999999999997 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377918+0000", 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9870000000000006e-05 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378036+0000", 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117105 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:713 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 7168~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.376982+0000", 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33281313600000001, 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001181849, 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 713 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376982+0000", 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376982+0000", 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376983+0000", 2024-09-21T23:16:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376985+0000", 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6170000000000001e-06 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376992+0000", 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.533e-06 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.376997+0000", 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4439999999999999e-06 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378068+0000", 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001070433 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378108+0000", 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0652999999999999e-05 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378164+0000", 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5755999999999999e-05 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:714 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 7680~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.377005+0000", 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.332790645, 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014023729999999999, 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 714 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377005+0000", 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377005+0000", 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377005+0000", 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377007+0000", 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0999999999999998e-06 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377014+0000", 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6059999999999999e-06 2024-09-21T23:16:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.377019+0000", 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1229999999999999e-06 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378201+0000", 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011823890000000001 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378285+0000", 2024-09-21T23:16:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3483e-05 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378407+0000", 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012229600000000001 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:720 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 14336~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.378970+0000", 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33082557899999998, 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010696010000000001, 2024-09-21T23:16:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 720 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378970+0000", 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378970+0000", 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378970+0000", 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378973+0000", 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4990000000000001e-06 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378979+0000", 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6229999999999999e-06 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.378984+0000", 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9679999999999997e-06 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379828+0000", 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084312700000000005 2024-09-21T23:16:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379916+0000", 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8116999999999997e-05 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.380040+0000", 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012381299999999999 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:721 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 15360~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.379009+0000", 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33078638900000001, 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012744239999999999, 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 721 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379009+0000", 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379009+0000", 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379010+0000", 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:16:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379012+0000", 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9939999999999999e-06 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379019+0000", 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5989999999999996e-06 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379025+0000", 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9159999999999998e-06 2024-09-21T23:16:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.380077+0000", 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010513930000000001 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.380162+0000", 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5080000000000005e-05 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.380284+0000", 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000121778 2024-09-21T23:16:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4894.0:722 3.12 3:4b0f3714:::rbd_data.10f1e9e481e5.0000000000000000:head [read 15872~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:07.379034+0000", 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33076149999999999, 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014213369999999999, 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4894", 2024-09-21T23:16:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2510029379", 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 722 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379034+0000", 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379034+0000", 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379035+0000", 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4799999999999998e-07 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379038+0000", 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.704e-06 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379046+0000", 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1640000000000001e-06 2024-09-21T23:16:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.379052+0000", 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.380322+0000", 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012705920000000001 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.380379+0000", 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6876000000000002e-05 2024-09-21T23:16:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.380455+0000", 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6123000000000003e-05 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:07.809 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:07.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:07.811 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:07.807+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:07.912 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:07.907+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:07.945 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-21T23:16:07.953 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:07.953 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:07.953 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:07.953 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:07.967 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:08.012 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:08.007+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:08.095 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:08.096 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:08.101 INFO:teuthology.orchestra.run.smithi170.stdout:2675 write 0xc2a4200 thru 0xc2b21ac (0xdfad bytes) 2024-09-21T23:16:08.102 INFO:teuthology.orchestra.run.smithi170.stdout:2676 read 0x684d600 thru 0x685cfdb (0xf9dc bytes) 2024-09-21T23:16:08.103 INFO:teuthology.orchestra.run.smithi170.stdout:2677 write 0xb30d200 thru 0xb314d55 (0x7b56 bytes) 2024-09-21T23:16:08.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:08.113 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:08.112+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:08.120 INFO:teuthology.orchestra.run.smithi170.stdout:2678 trunc from 0xc2b2200 to 0xdfad400 2024-09-21T23:16:08.208 INFO:teuthology.orchestra.run.smithi170.stdout:2680 write 0x90a0400 thru 0x90aba88 (0xb689 bytes) 2024-09-21T23:16:08.209 INFO:teuthology.orchestra.run.smithi170.stdout:2684 punch from 0x68a0000 to 0x68a2e00, (0x2e00 bytes) 2024-09-21T23:16:08.214 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:08.212+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:08.218 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:16:08.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:16:08.242 INFO:teuthology.orchestra.run.smithi170.stdout:2685 clone 16 order 23 su 524288 sc 12 2024-09-21T23:16:08.253 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:16:08.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:16:08.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:08.270 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:08.270 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:08.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:08.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:08.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.270 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:16:08.270 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.924744507, 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:16:08.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:16:08.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:08.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:16:08.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.278 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:16:08.278 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:08.278 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.157102146, 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:08.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:16:08.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:16:08.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.777792064000003, 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:16:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:16:08.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.650575052000001, 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.290 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:16:08.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:16:08.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:16:08.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.735811321, 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:16:08.297 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:16:08.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:16:08.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:16:08.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.280858555000002, 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:08.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:16:08.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:16:08.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:16:08.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:16:08.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:16:08.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:16:08.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 47.430832172000002, 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:16:08.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 47.430790754, 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:16:08.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:16:08.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 47.248704828000001, 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:08.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:16:08.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:16:08.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:16:08.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 47.106762484999997, 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.950896190000002, 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:08.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:08.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:16:08.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:08.346 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.788165913, 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:08.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:16:08.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.268740530000002, 2024-09-21T23:16:08.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:16:08.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:08.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:16:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:16:08.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.268612054000002, 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:16:08.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:16:08.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:08.368 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.267650349999997, 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:16:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.266639138999999, 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:16:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.265557192000003, 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:16:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.388978934000001, 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:16:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.388639363999999, 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:16:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:16:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:16:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:596 3.12 3:4d026f6e:::rbd_data.10f1dc9013.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.318924353, 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078799430000000004, 2024-09-21T23:16:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 596 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951020+0000", 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-06 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951929+0000", 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090880699999999997 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951965+0000", 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5734e-05 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951982+0000", 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6806000000000001e-05 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952135+0000", 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015325899999999999 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952560+0000", 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042476500000000002 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.953181+0000", 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062147299999999999 2024-09-21T23:16:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.957005+0000", 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038238 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958342+0000", 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001336622 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958557+0000", 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021509599999999999 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958897+0000", 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034014100000000002 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:08.404 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:08.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:08.408 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:08.404+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:08.507 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:08.503+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:08.546 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:08.563 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:08.564 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:08.573 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:08.574 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:08.581 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:08.581 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:08.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:08.603 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:08.603 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:08.607 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:08.603+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:08.623 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:08.624 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:08.644 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:08.644 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:08.706 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:08.706 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:08.708 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:08.708+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:08.713 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:16:08.722 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:08.732 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-21T23:16:08.216933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":234380364,"num_objects":451,"num_object_clones":115,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":143,"num_read":89928,"num_read_kb":321133,"num_write":9779,"num_write_kb":52724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":1749870,"num_keys_recovered":60,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8382,"ondisk_log_size":8382,"up":82,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":213260,"kb_used_data":44336,"kb_used_omap":10,"kb_used_meta":168821,"kb_avail":377274100,"statfs":{"total":386547056640,"available":386328678400,"internally_reserved":0,"allocated":45400064,"data_stored":52724802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":172873332},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38811230,"num_objects":43,"num_object_clones":32,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":-50,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":2090,"num_read_kb":5020,"num_write":126,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":595116,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.062731"},"pg_stats":[{"pgid":"3.1f","version":"63'175","reported_seq":392,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.201320+0000","last_change":"2024-09-21T23:16:05.201211+0000","last_active":"2024-09-21T23:16:05.201211+0000","last_peered":"2024-09-21T23:16:04.648784+0000","last_clean":"2024-09-21T23:15:08.357378+0000","last_became_active":"2024-09-21T23:16:04.202564+0000","last_became_peered":"2024-09-21T23:16:04.202564+0000","last_unstale":"2024-09-21T23:16:05.201320+0000","last_undegraded":"2024-09-21T23:16:05.201320+0000","last_fullsized":"2024-09-21T23:16:05.201320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":6582836,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":425,"num_read_kb":473,"num_write":222,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":14388,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"59'181","reported_seq":996,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.527236+0000","last_change":"2024-09-21T23:15:35.968961+0000","last_active":"2024-09-21T23:16:04.527236+0000","last_peered":"2024-09-21T23:16:04.527236+0000","last_clean":"2024-09-21T23:16:04.527236+0000","last_became_active":"2024-09-21T23:15:35.968136+0000","last_became_peered":"2024-09-21T23:15:35.968136+0000","last_unstale":"2024-09-21T23:16:04.527236+0000","last_undegraded":"2024-09-21T23:16:04.527236+0000","last_fullsized":"2024-09-21T23:16:04.527236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:59.548525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":8337956,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":698,"num_read_kb":9271,"num_write":173,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"59'81","reported_seq":1204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.525067+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:16:04.525067+0000","last_peered":"2024-09-21T23:16:04.525067+0000","last_clean":"2024-09-21T23:16:04.525067+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:16:04.525067+0000","last_undegraded":"2024-09-21T23:16:04.525067+0000","last_fullsized":"2024-09-21T23:16:04.525067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:47:53.671106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":4360704,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1031,"num_read_kb":5876,"num_write":82,"num_write_kb":1980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'217","reported_seq":2391,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.766882+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:16:04.766882+0000","last_peered":"2024-09-21T23:16:04.766882+0000","last_clean":"2024-09-21T23:16:04.766882+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:16:04.766882+0000","last_undegraded":"2024-09-21T23:16:04.766882+0000","last_fullsized":"2024-09-21T23:16:04.766882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:01:28.334770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":10842176,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":2295,"num_read_kb":16616,"num_write":202,"num_write_kb":5067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"58'231","reported_seq":2023,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.206485+0000","last_change":"2024-09-21T23:16:05.206485+0000","last_active":"2024-09-21T23:16:03.163194+0000","last_peered":"2024-09-21T23:16:01.151190+0000","last_clean":"2024-09-21T23:15:08.357275+0000","last_became_active":"2024-09-21T23:15:11.742173+0000","last_became_peered":"2024-09-21T23:15:11.742173+0000","last_unstale":"2024-09-21T23:16:05.206485+0000","last_undegraded":"2024-09-21T23:16:05.206485+0000","last_fullsized":"2024-09-21T23:16:05.206485+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":5812766,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":2685,"num_read_kb":11595,"num_write":252,"num_write_kb":874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"63'280","reported_seq":5173,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.740820+0000","last_change":"2024-09-21T23:15:35.969265+0000","last_active":"2024-09-21T23:16:04.740820+0000","last_peered":"2024-09-21T23:16:04.740820+0000","last_clean":"2024-09-21T23:16:04.740820+0000","last_became_active":"2024-09-21T23:15:35.967793+0000","last_became_peered":"2024-09-21T23:15:35.967793+0000","last_unstale":"2024-09-21T23:16:04.740820+0000","last_undegraded":"2024-09-21T23:16:04.740820+0000","last_fullsized":"2024-09-21T23:16:04.740820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:02:28.996499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":4649507,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":6577,"num_read_kb":16480,"num_write":207,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":39443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"58'114","reported_seq":1120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.164122+0000","last_change":"2024-09-21T23:15:32.983007+0000","last_active":"2024-09-21T23:16:04.164122+0000","last_peered":"2024-09-21T23:16:04.164122+0000","last_clean":"2024-09-21T23:16:04.164122+0000","last_became_active":"2024-09-21T23:15:32.948191+0000","last_became_peered":"2024-09-21T23:15:32.948191+0000","last_unstale":"2024-09-21T23:16:04.164122+0000","last_undegraded":"2024-09-21T23:16:04.164122+0000","last_fullsized":"2024-09-21T23:16:04.164122+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:34:04.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":4866048,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":876,"num_read_kb":6282,"num_write":113,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"63'321","reported_seq":2016,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.879668+0000","last_change":"2024-09-21T23:16:04.865800+0000","last_active":"2024-09-21T23:16:04.879668+0000","last_peered":"2024-09-21T23:16:04.879668+0000","last_clean":"2024-09-21T23:15:10.369468+0000","last_became_active":"2024-09-21T23:16:04.210375+0000","last_became_peered":"2024-09-21T23:16:04.210375+0000","last_unstale":"2024-09-21T23:16:04.879668+0000","last_undegraded":"2024-09-21T23:16:04.167066+0000","last_fullsized":"2024-09-21T23:16:04.160794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":10488892,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":2675,"num_read_kb":13513,"num_write":481,"num_write_kb":2839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"63'149","reported_seq":490,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.201797+0000","last_change":"2024-09-21T23:16:05.201645+0000","last_active":"2024-09-21T23:16:05.201645+0000","last_peered":"2024-09-21T23:16:04.750843+0000","last_clean":"2024-09-21T23:15:09.361363+0000","last_became_active":"2024-09-21T23:16:04.202539+0000","last_became_peered":"2024-09-21T23:16:04.202539+0000","last_unstale":"2024-09-21T23:16:05.201797+0000","last_undegraded":"2024-09-21T23:16:05.201797+0000","last_fullsized":"2024-09-21T23:16:05.201797+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":11392064,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":226,"num_read_kb":1884,"num_write":146,"num_write_kb":681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":25664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"59'274","reported_seq":1560,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.510157+0000","last_change":"2024-09-21T23:15:35.965248+0000","last_active":"2024-09-21T23:16:04.510157+0000","last_peered":"2024-09-21T23:16:04.510157+0000","last_clean":"2024-09-21T23:16:04.510157+0000","last_became_active":"2024-09-21T23:15:35.963512+0000","last_became_peered":"2024-09-21T23:15:35.963512+0000","last_unstale":"2024-09-21T23:16:04.510157+0000","last_undegraded":"2024-09-21T23:16:04.510157+0000","last_fullsized":"2024-09-21T23:16:04.510157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:44:51.065083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":15697570,"num_objects":25,"num_object_clones":9,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1899,"num_read_kb":6760,"num_write":377,"num_write_kb":2104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":94370,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"58'115","reported_seq":2909,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.207500+0000","last_change":"2024-09-21T23:16:05.207500+0000","last_active":"2024-09-21T23:16:03.162957+0000","last_peered":"2024-09-21T23:16:01.151932+0000","last_clean":"2024-09-21T23:15:08.790498+0000","last_became_active":"2024-09-21T23:15:11.743088+0000","last_became_peered":"2024-09-21T23:15:11.743088+0000","last_unstale":"2024-09-21T23:16:05.207500+0000","last_undegraded":"2024-09-21T23:16:05.207500+0000","last_fullsized":"2024-09-21T23:16:05.207500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898304,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2710,"num_read_kb":24295,"num_write":114,"num_write_kb":2601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"63'483","reported_seq":1788,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.528567+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:16:04.528567+0000","last_peered":"2024-09-21T23:16:04.528567+0000","last_clean":"2024-09-21T23:16:04.528567+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:16:04.528567+0000","last_undegraded":"2024-09-21T23:16:04.528567+0000","last_fullsized":"2024-09-21T23:16:04.528567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:32.206028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260319,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":3003,"num_read_kb":11616,"num_write":727,"num_write_kb":1391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.163843+0000","last_change":"2024-09-21T23:15:06.350164+0000","last_active":"2024-09-21T23:16:04.163843+0000","last_peered":"2024-09-21T23:16:04.163843+0000","last_clean":"2024-09-21T23:16:04.163843+0000","last_became_active":"2024-09-21T23:15:06.349271+0000","last_became_peered":"2024-09-21T23:15:06.349271+0000","last_unstale":"2024-09-21T23:16:04.163843+0000","last_undegraded":"2024-09-21T23:16:04.163843+0000","last_fullsized":"2024-09-21T23:16:04.163843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:56.638782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"63'157","reported_seq":656,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.208419+0000","last_change":"2024-09-21T23:16:05.207857+0000","last_active":"2024-09-21T23:16:05.207857+0000","last_peered":"2024-09-21T23:16:04.750340+0000","last_clean":"2024-09-21T23:16:01.924947+0000","last_became_active":"2024-09-21T23:16:04.199145+0000","last_became_peered":"2024-09-21T23:16:04.199145+0000","last_unstale":"2024-09-21T23:16:05.208419+0000","last_undegraded":"2024-09-21T23:16:05.208419+0000","last_fullsized":"2024-09-21T23:16:05.208419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":20174888,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":355,"num_read_kb":2331,"num_write":154,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.189267+0000","last_change":"2024-09-21T23:16:04.189267+0000","last_active":"2024-09-21T23:16:04.189267+0000","last_peered":"2024-09-21T23:16:04.189267+0000","last_clean":"2024-09-21T23:16:04.189267+0000","last_became_active":"2024-09-21T23:16:04.189021+0000","last_became_peered":"2024-09-21T23:16:04.189021+0000","last_unstale":"2024-09-21T23:16:04.189267+0000","last_undegraded":"2024-09-21T23:16:04.189267+0000","last_fullsized":"2024-09-21T23:16:04.189267+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:19.263980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"58'1465","reported_seq":4149,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.162812+0000","last_change":"2024-09-21T23:15:35.969593+0000","last_active":"2024-09-21T23:16:04.162812+0000","last_peered":"2024-09-21T23:16:04.162812+0000","last_clean":"2024-09-21T23:16:04.162812+0000","last_became_active":"2024-09-21T23:15:35.967156+0000","last_became_peered":"2024-09-21T23:15:35.967156+0000","last_unstale":"2024-09-21T23:16:04.162812+0000","last_undegraded":"2024-09-21T23:16:04.162812+0000","last_fullsized":"2024-09-21T23:16:04.162812+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1465,"log_dups_size":0,"ondisk_log_size":1465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:27.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":7646122,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":9741,"num_read_kb":28640,"num_write":2644,"num_write_kb":5251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":247168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.207178+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:16:05.207178+0000","last_peered":"2024-09-21T23:16:05.207178+0000","last_clean":"2024-09-21T23:16:05.207178+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:16:05.207178+0000","last_undegraded":"2024-09-21T23:16:05.207178+0000","last_fullsized":"2024-09-21T23:16:05.207178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"59'116","reported_seq":741,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.527095+0000","last_change":"2024-09-21T23:16:04.314544+0000","last_active":"2024-09-21T23:16:04.527095+0000","last_peered":"2024-09-21T23:16:04.527095+0000","last_clean":"2024-09-21T23:16:01.928682+0000","last_became_active":"2024-09-21T23:16:04.198318+0000","last_became_peered":"2024-09-21T23:16:04.198318+0000","last_unstale":"2024-09-21T23:16:04.527095+0000","last_undegraded":"2024-09-21T23:16:04.527095+0000","last_fullsized":"2024-09-21T23:16:04.161145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":3896832,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":985,"num_read_kb":1563,"num_write":99,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":52736,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":11}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.206345+0000","last_change":"2024-09-21T23:16:04.198942+0000","last_active":"2024-09-21T23:16:05.206345+0000","last_peered":"2024-09-21T23:16:05.206345+0000","last_clean":"2024-09-21T23:16:05.206345+0000","last_became_active":"2024-09-21T23:16:04.198333+0000","last_became_peered":"2024-09-21T23:16:04.198333+0000","last_unstale":"2024-09-21T23:16:05.206345+0000","last_undegraded":"2024-09-21T23:16:05.206345+0000","last_fullsized":"2024-09-21T23:16:05.206345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:59.745128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"59'280","reported_seq":2299,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.513552+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:16:04.513552+0000","last_peered":"2024-09-21T23:16:04.513552+0000","last_clean":"2024-09-21T23:16:04.513552+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:16:04.513552+0000","last_undegraded":"2024-09-21T23:16:04.513552+0000","last_fullsized":"2024-09-21T23:16:04.513552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:31.015952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":9773854,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":3265,"num_read_kb":14249,"num_write":328,"num_write_kb":1511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'188","reported_seq":1803,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.199908+0000","last_change":"2024-09-21T23:16:04.200078+0000","last_active":"2024-09-21T23:16:04.199908+0000","last_peered":"2024-09-21T23:16:04.199908+0000","last_clean":"2024-09-21T23:15:10.359509+0000","last_became_active":"2024-09-21T23:16:04.199385+0000","last_became_peered":"2024-09-21T23:16:04.199385+0000","last_unstale":"2024-09-21T23:16:04.199908+0000","last_undegraded":"2024-09-21T23:16:04.165390+0000","last_fullsized":"2024-09-21T23:16:04.164568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":7304734,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":1643,"num_read_kb":10737,"num_write":197,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":13}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.205303+0000","last_change":"2024-09-21T23:16:04.205303+0000","last_active":"2024-09-21T23:16:04.205303+0000","last_peered":"2024-09-21T23:16:04.205303+0000","last_clean":"2024-09-21T23:16:04.205303+0000","last_became_active":"2024-09-21T23:16:04.181322+0000","last_became_peered":"2024-09-21T23:16:04.181322+0000","last_unstale":"2024-09-21T23:16:04.205303+0000","last_undegraded":"2024-09-21T23:16:04.205303+0000","last_fullsized":"2024-09-21T23:16:04.205303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:04.294981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'163","reported_seq":1840,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.512692+0000","last_change":"2024-09-21T23:15:35.969320+0000","last_active":"2024-09-21T23:16:04.512692+0000","last_peered":"2024-09-21T23:16:04.512692+0000","last_clean":"2024-09-21T23:16:04.512692+0000","last_became_active":"2024-09-21T23:15:35.968596+0000","last_became_peered":"2024-09-21T23:15:35.968596+0000","last_unstale":"2024-09-21T23:16:04.512692+0000","last_undegraded":"2024-09-21T23:16:04.512692+0000","last_fullsized":"2024-09-21T23:16:04.512692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:28:40.180849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":3327604,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1552,"num_read_kb":6767,"num_write":158,"num_write_kb":681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":29200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.197581+0000","last_change":"2024-09-21T23:16:04.197581+0000","last_active":"2024-09-21T23:16:04.197581+0000","last_peered":"2024-09-21T23:16:04.197581+0000","last_clean":"2024-09-21T23:16:04.197581+0000","last_became_active":"2024-09-21T23:16:04.196985+0000","last_became_peered":"2024-09-21T23:16:04.196985+0000","last_unstale":"2024-09-21T23:16:04.197581+0000","last_undegraded":"2024-09-21T23:16:04.197581+0000","last_fullsized":"2024-09-21T23:16:04.197581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:24:15.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'193","reported_seq":1352,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.207147+0000","last_change":"2024-09-21T23:16:05.207147+0000","last_active":"2024-09-21T23:16:03.161294+0000","last_peered":"2024-09-21T23:16:01.150966+0000","last_clean":"2024-09-21T23:15:08.563705+0000","last_became_active":"2024-09-21T23:15:11.742147+0000","last_became_peered":"2024-09-21T23:15:11.742147+0000","last_unstale":"2024-09-21T23:16:05.207147+0000","last_undegraded":"2024-09-21T23:16:05.207147+0000","last_fullsized":"2024-09-21T23:16:05.207147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":3952676,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":1107,"num_read_kb":6575,"num_write":197,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.201232+0000","last_change":"2024-09-21T23:16:04.207675+0000","last_active":"2024-09-21T23:16:05.201232+0000","last_peered":"2024-09-21T23:16:05.201232+0000","last_clean":"2024-09-21T23:16:05.201232+0000","last_became_active":"2024-09-21T23:16:04.206739+0000","last_became_peered":"2024-09-21T23:16:04.206739+0000","last_unstale":"2024-09-21T23:16:05.201232+0000","last_undegraded":"2024-09-21T23:16:05.201232+0000","last_fullsized":"2024-09-21T23:16:05.201232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:12.680088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"63'164","reported_seq":598,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.755609+0000","last_change":"2024-09-21T23:15:33.016643+0000","last_active":"2024-09-21T23:16:04.755609+0000","last_peered":"2024-09-21T23:16:04.755609+0000","last_clean":"2024-09-21T23:16:04.755609+0000","last_became_active":"2024-09-21T23:15:32.955127+0000","last_became_peered":"2024-09-21T23:15:32.955127+0000","last_unstale":"2024-09-21T23:16:04.755609+0000","last_undegraded":"2024-09-21T23:16:04.755609+0000","last_fullsized":"2024-09-21T23:16:04.755609+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:50.308037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":13215350,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":387,"num_read_kb":3830,"num_write":169,"num_write_kb":1817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":10301,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.155775+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:16:04.155775+0000","last_peered":"2024-09-21T23:16:04.155775+0000","last_clean":"2024-09-21T23:16:04.155775+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:16:04.155775+0000","last_undegraded":"2024-09-21T23:16:04.155775+0000","last_fullsized":"2024-09-21T23:16:04.155775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.207669+0000","last_change":"2024-09-21T23:16:04.207237+0000","last_active":"2024-09-21T23:16:05.207669+0000","last_peered":"2024-09-21T23:16:05.207669+0000","last_clean":"2024-09-21T23:16:05.207669+0000","last_became_active":"2024-09-21T23:16:04.206117+0000","last_became_peered":"2024-09-21T23:16:04.206117+0000","last_unstale":"2024-09-21T23:16:05.207669+0000","last_undegraded":"2024-09-21T23:16:05.207669+0000","last_fullsized":"2024-09-21T23:16:05.207669+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:26:34.280570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"59'508","reported_seq":9814,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.527080+0000","last_change":"2024-09-21T23:15:35.968122+0000","last_active":"2024-09-21T23:16:04.527080+0000","last_peered":"2024-09-21T23:16:04.527080+0000","last_clean":"2024-09-21T23:16:04.527080+0000","last_became_active":"2024-09-21T23:15:35.967154+0000","last_became_peered":"2024-09-21T23:15:35.967154+0000","last_unstale":"2024-09-21T23:16:04.527080+0000","last_undegraded":"2024-09-21T23:16:04.527080+0000","last_fullsized":"2024-09-21T23:16:04.527080+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:41.598133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":3641872,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":10525,"num_read_kb":14824,"num_write":300,"num_write_kb":1356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":15888,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"59'293","reported_seq":2856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.515765+0000","last_change":"2024-09-21T23:15:03.325366+0000","last_active":"2024-09-21T23:16:04.515765+0000","last_peered":"2024-09-21T23:16:04.515765+0000","last_clean":"2024-09-21T23:16:04.515765+0000","last_became_active":"2024-09-21T23:15:01.329082+0000","last_became_peered":"2024-09-21T23:15:01.329082+0000","last_unstale":"2024-09-21T23:16:04.515765+0000","last_undegraded":"2024-09-21T23:16:04.515765+0000","last_fullsized":"2024-09-21T23:16:04.515765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:46.526439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":5206528,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":4289,"num_read_kb":7869,"num_write":214,"num_write_kb":865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"59'127","reported_seq":442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.523750+0000","last_change":"2024-09-21T23:15:33.025560+0000","last_active":"2024-09-21T23:16:04.523750+0000","last_peered":"2024-09-21T23:16:04.523750+0000","last_clean":"2024-09-21T23:16:04.523750+0000","last_became_active":"2024-09-21T23:15:32.949370+0000","last_became_peered":"2024-09-21T23:15:32.949370+0000","last_unstale":"2024-09-21T23:16:04.523750+0000","last_undegraded":"2024-09-21T23:16:04.523750+0000","last_fullsized":"2024-09-21T23:16:04.523750+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:22.683465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988624,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":195,"num_read_kb":1010,"num_write":124,"num_write_kb":643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":8192,"num_keys_recovered":3,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"63'314","reported_seq":3708,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.657118+0000","last_change":"2024-09-21T23:16:04.206606+0000","last_active":"2024-09-21T23:16:04.657118+0000","last_peered":"2024-09-21T23:16:04.657118+0000","last_clean":"2024-09-21T23:16:01.975089+0000","last_became_active":"2024-09-21T23:16:04.205124+0000","last_became_peered":"2024-09-21T23:16:04.205124+0000","last_unstale":"2024-09-21T23:16:04.657118+0000","last_undegraded":"2024-09-21T23:16:04.165963+0000","last_fullsized":"2024-09-21T23:16:04.162982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":3923487,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":5208,"num_read_kb":13998,"num_write":220,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":94208,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[4],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,4","objects":17},{"shards":"4","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"59'202","reported_seq":667,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.571875+0000","last_change":"2024-09-21T23:16:04.572590+0000","last_active":"2024-09-21T23:16:04.571875+0000","last_peered":"2024-09-21T23:16:04.571875+0000","last_clean":"2024-09-21T23:15:10.367359+0000","last_became_active":"2024-09-21T23:16:04.198934+0000","last_became_peered":"2024-09-21T23:16:04.198934+0000","last_unstale":"2024-09-21T23:16:04.571875+0000","last_undegraded":"2024-09-21T23:16:04.156963+0000","last_fullsized":"2024-09-21T23:16:04.156016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":5001773,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":336,"num_read_kb":993,"num_write":210,"num_write_kb":1082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"5","objects":13}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'182","reported_seq":3236,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.513061+0000","last_change":"2024-09-21T23:15:06.357124+0000","last_active":"2024-09-21T23:16:04.513061+0000","last_peered":"2024-09-21T23:16:04.513061+0000","last_clean":"2024-09-21T23:16:04.513061+0000","last_became_active":"2024-09-21T23:15:06.356745+0000","last_became_peered":"2024-09-21T23:15:06.356745+0000","last_unstale":"2024-09-21T23:16:04.513061+0000","last_undegraded":"2024-09-21T23:16:04.513061+0000","last_fullsized":"2024-09-21T23:16:04.513061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:15.067781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5444218,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":3235,"num_read_kb":23473,"num_write":217,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"58'154","reported_seq":1881,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.155909+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:16:04.155909+0000","last_peered":"2024-09-21T23:16:04.155909+0000","last_clean":"2024-09-21T23:16:04.155909+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:16:04.155909+0000","last_undegraded":"2024-09-21T23:16:04.155909+0000","last_fullsized":"2024-09-21T23:16:04.155909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:08.946701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":4626432,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":2526,"num_read_kb":7546,"num_write":126,"num_write_kb":693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"58'138","reported_seq":2027,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.383573+0000","last_change":"2024-09-21T23:16:04.384089+0000","last_active":"2024-09-21T23:16:04.383573+0000","last_peered":"2024-09-21T23:16:04.383573+0000","last_clean":"2024-09-21T23:15:10.367429+0000","last_became_active":"2024-09-21T23:16:04.196883+0000","last_became_peered":"2024-09-21T23:16:04.196883+0000","last_unstale":"2024-09-21T23:16:04.383573+0000","last_undegraded":"2024-09-21T23:16:04.383573+0000","last_fullsized":"2024-09-21T23:16:04.156010+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":5008400,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":2053,"num_read_kb":6536,"num_write":117,"num_write_kb":494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"5","objects":9}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"59'174","reported_seq":1553,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.512598+0000","last_change":"2024-09-21T23:16:04.206447+0000","last_active":"2024-09-21T23:16:04.512598+0000","last_peered":"2024-09-21T23:16:04.512598+0000","last_clean":"2024-09-21T23:15:10.367211+0000","last_became_active":"2024-09-21T23:16:04.206033+0000","last_became_peered":"2024-09-21T23:16:04.206033+0000","last_unstale":"2024-09-21T23:16:04.512598+0000","last_undegraded":"2024-09-21T23:16:04.156677+0000","last_fullsized":"2024-09-21T23:16:04.155552+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":5904896,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1279,"num_read_kb":9175,"num_write":168,"num_write_kb":1018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":14}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"59'325","reported_seq":1798,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.516624+0000","last_change":"2024-09-21T23:16:04.444160+0000","last_active":"2024-09-21T23:16:04.516624+0000","last_peered":"2024-09-21T23:16:04.516624+0000","last_clean":"2024-09-21T23:15:10.359214+0000","last_became_active":"2024-09-21T23:16:04.195755+0000","last_became_peered":"2024-09-21T23:16:04.195755+0000","last_unstale":"2024-09-21T23:16:04.516624+0000","last_undegraded":"2024-09-21T23:16:04.516624+0000","last_fullsized":"2024-09-21T23:16:04.162831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2856485,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":2459,"num_read_kb":5407,"num_write":489,"num_write_kb":933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":["4","0"],"object_location_counts":[{"shards":"4","objects":14}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"63'337","reported_seq":4648,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.202268+0000","last_change":"2024-09-21T23:16:05.200483+0000","last_active":"2024-09-21T23:16:05.200483+0000","last_peered":"2024-09-21T23:16:04.675100+0000","last_clean":"2024-09-21T23:16:01.157282+0000","last_became_active":"2024-09-21T23:16:04.206272+0000","last_became_peered":"2024-09-21T23:16:04.206272+0000","last_unstale":"2024-09-21T23:16:05.202268+0000","last_undegraded":"2024-09-21T23:16:05.202268+0000","last_fullsized":"2024-09-21T23:16:05.202268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":10283580,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":6198,"num_read_kb":14619,"num_write":239,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":154112,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'161","reported_seq":7605,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.524795+0000","last_change":"2024-09-21T23:15:06.345498+0000","last_active":"2024-09-21T23:16:04.524795+0000","last_peered":"2024-09-21T23:16:04.524795+0000","last_clean":"2024-09-21T23:16:04.524795+0000","last_became_active":"2024-09-21T23:15:06.344491+0000","last_became_peered":"2024-09-21T23:15:06.344491+0000","last_unstale":"2024-09-21T23:16:04.524795+0000","last_undegraded":"2024-09-21T23:16:04.524795+0000","last_fullsized":"2024-09-21T23:16:04.524795+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:30:45.230225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7422480,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":7374,"num_read_kb":16230,"num_write":152,"num_write_kb":763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":233789977,"num_objects":447,"num_object_clones":115,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":143,"num_read":89812,"num_read_kb":321033,"num_write":9618,"num_write_kb":50306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":488,"num_bytes_recovered":1159483,"num_keys_recovered":60,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43405312,"data_stored":52380869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":8262,"ondisk_log_size":8262,"up":64,"acting":56,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52084,"kb_used_data":12572,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319756,"statfs":{"total":96636764160,"available":96583430144,"internally_reserved":0,"allocated":12873728,"data_stored":16126365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":15672,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":16048128,"data_stored":19027088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607572,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607572,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61932,"kb_used_data":8532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309908,"statfs":{"total":96636764160,"available":96573345792,"internally_reserved":0,"allocated":8736768,"data_stored":9395283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607573,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005058,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":7560,"kb_used_omap":5,"kb_used_meta":32314,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":7741440,"data_stored":8176066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":33089829},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6410240,"data_stored":7286773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1794048,"data_stored":1745027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":8516971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15503360,"data_stored":18895084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12259328,"data_stored":15936998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:16:08.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:16:08.810 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:08.803+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:08.892 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:08.892 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:08.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:08.910 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:08.903+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:09.010 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:09.007+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:09.041 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:09.041 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:09.055 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:09.074 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:09.075 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:16:09.094 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-21T23:16:08.865200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":234380364,"num_objects":451,"num_object_clones":115,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":143,"num_read":89928,"num_read_kb":321133,"num_write":9779,"num_write_kb":52724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":1749870,"num_keys_recovered":60,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8382,"ondisk_log_size":8382,"up":82,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":213260,"kb_used_data":44336,"kb_used_omap":10,"kb_used_meta":168821,"kb_avail":377274100,"statfs":{"total":386547056640,"available":386328678400,"internally_reserved":0,"allocated":45400064,"data_stored":52724802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":172873332},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38811230,"num_objects":43,"num_object_clones":32,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":-50,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":2090,"num_read_kb":5020,"num_write":126,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":595116,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001326"},"pg_stats":[{"pgid":"3.1f","version":"63'175","reported_seq":392,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.201320+0000","last_change":"2024-09-21T23:16:05.201211+0000","last_active":"2024-09-21T23:16:05.201211+0000","last_peered":"2024-09-21T23:16:04.648784+0000","last_clean":"2024-09-21T23:15:08.357378+0000","last_became_active":"2024-09-21T23:16:04.202564+0000","last_became_peered":"2024-09-21T23:16:04.202564+0000","last_unstale":"2024-09-21T23:16:05.201320+0000","last_undegraded":"2024-09-21T23:16:05.201320+0000","last_fullsized":"2024-09-21T23:16:05.201320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":6582836,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":425,"num_read_kb":473,"num_write":222,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":14388,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"59'181","reported_seq":996,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.527236+0000","last_change":"2024-09-21T23:15:35.968961+0000","last_active":"2024-09-21T23:16:04.527236+0000","last_peered":"2024-09-21T23:16:04.527236+0000","last_clean":"2024-09-21T23:16:04.527236+0000","last_became_active":"2024-09-21T23:15:35.968136+0000","last_became_peered":"2024-09-21T23:15:35.968136+0000","last_unstale":"2024-09-21T23:16:04.527236+0000","last_undegraded":"2024-09-21T23:16:04.527236+0000","last_fullsized":"2024-09-21T23:16:04.527236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:59.548525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":8337956,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":698,"num_read_kb":9271,"num_write":173,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"59'81","reported_seq":1204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.525067+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:16:04.525067+0000","last_peered":"2024-09-21T23:16:04.525067+0000","last_clean":"2024-09-21T23:16:04.525067+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:16:04.525067+0000","last_undegraded":"2024-09-21T23:16:04.525067+0000","last_fullsized":"2024-09-21T23:16:04.525067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:47:53.671106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":4360704,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1031,"num_read_kb":5876,"num_write":82,"num_write_kb":1980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'217","reported_seq":2391,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.766882+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:16:04.766882+0000","last_peered":"2024-09-21T23:16:04.766882+0000","last_clean":"2024-09-21T23:16:04.766882+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:16:04.766882+0000","last_undegraded":"2024-09-21T23:16:04.766882+0000","last_fullsized":"2024-09-21T23:16:04.766882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:01:28.334770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":10842176,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":2295,"num_read_kb":16616,"num_write":202,"num_write_kb":5067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"58'231","reported_seq":2023,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.206485+0000","last_change":"2024-09-21T23:16:05.206485+0000","last_active":"2024-09-21T23:16:03.163194+0000","last_peered":"2024-09-21T23:16:01.151190+0000","last_clean":"2024-09-21T23:15:08.357275+0000","last_became_active":"2024-09-21T23:15:11.742173+0000","last_became_peered":"2024-09-21T23:15:11.742173+0000","last_unstale":"2024-09-21T23:16:05.206485+0000","last_undegraded":"2024-09-21T23:16:05.206485+0000","last_fullsized":"2024-09-21T23:16:05.206485+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":5812766,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":2685,"num_read_kb":11595,"num_write":252,"num_write_kb":874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"63'280","reported_seq":5173,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.740820+0000","last_change":"2024-09-21T23:15:35.969265+0000","last_active":"2024-09-21T23:16:04.740820+0000","last_peered":"2024-09-21T23:16:04.740820+0000","last_clean":"2024-09-21T23:16:04.740820+0000","last_became_active":"2024-09-21T23:15:35.967793+0000","last_became_peered":"2024-09-21T23:15:35.967793+0000","last_unstale":"2024-09-21T23:16:04.740820+0000","last_undegraded":"2024-09-21T23:16:04.740820+0000","last_fullsized":"2024-09-21T23:16:04.740820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:02:28.996499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":4649507,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":6577,"num_read_kb":16480,"num_write":207,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":39443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"58'114","reported_seq":1120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.164122+0000","last_change":"2024-09-21T23:15:32.983007+0000","last_active":"2024-09-21T23:16:04.164122+0000","last_peered":"2024-09-21T23:16:04.164122+0000","last_clean":"2024-09-21T23:16:04.164122+0000","last_became_active":"2024-09-21T23:15:32.948191+0000","last_became_peered":"2024-09-21T23:15:32.948191+0000","last_unstale":"2024-09-21T23:16:04.164122+0000","last_undegraded":"2024-09-21T23:16:04.164122+0000","last_fullsized":"2024-09-21T23:16:04.164122+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:34:04.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":4866048,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":876,"num_read_kb":6282,"num_write":113,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"63'321","reported_seq":2016,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.879668+0000","last_change":"2024-09-21T23:16:04.865800+0000","last_active":"2024-09-21T23:16:04.879668+0000","last_peered":"2024-09-21T23:16:04.879668+0000","last_clean":"2024-09-21T23:15:10.369468+0000","last_became_active":"2024-09-21T23:16:04.210375+0000","last_became_peered":"2024-09-21T23:16:04.210375+0000","last_unstale":"2024-09-21T23:16:04.879668+0000","last_undegraded":"2024-09-21T23:16:04.167066+0000","last_fullsized":"2024-09-21T23:16:04.160794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":10488892,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":2675,"num_read_kb":13513,"num_write":481,"num_write_kb":2839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"63'149","reported_seq":490,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.201797+0000","last_change":"2024-09-21T23:16:05.201645+0000","last_active":"2024-09-21T23:16:05.201645+0000","last_peered":"2024-09-21T23:16:04.750843+0000","last_clean":"2024-09-21T23:15:09.361363+0000","last_became_active":"2024-09-21T23:16:04.202539+0000","last_became_peered":"2024-09-21T23:16:04.202539+0000","last_unstale":"2024-09-21T23:16:05.201797+0000","last_undegraded":"2024-09-21T23:16:05.201797+0000","last_fullsized":"2024-09-21T23:16:05.201797+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":11392064,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":226,"num_read_kb":1884,"num_write":146,"num_write_kb":681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":25664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"59'274","reported_seq":1560,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.510157+0000","last_change":"2024-09-21T23:15:35.965248+0000","last_active":"2024-09-21T23:16:04.510157+0000","last_peered":"2024-09-21T23:16:04.510157+0000","last_clean":"2024-09-21T23:16:04.510157+0000","last_became_active":"2024-09-21T23:15:35.963512+0000","last_became_peered":"2024-09-21T23:15:35.963512+0000","last_unstale":"2024-09-21T23:16:04.510157+0000","last_undegraded":"2024-09-21T23:16:04.510157+0000","last_fullsized":"2024-09-21T23:16:04.510157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:44:51.065083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":15697570,"num_objects":25,"num_object_clones":9,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1899,"num_read_kb":6760,"num_write":377,"num_write_kb":2104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":94370,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"58'115","reported_seq":2909,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.207500+0000","last_change":"2024-09-21T23:16:05.207500+0000","last_active":"2024-09-21T23:16:03.162957+0000","last_peered":"2024-09-21T23:16:01.151932+0000","last_clean":"2024-09-21T23:15:08.790498+0000","last_became_active":"2024-09-21T23:15:11.743088+0000","last_became_peered":"2024-09-21T23:15:11.743088+0000","last_unstale":"2024-09-21T23:16:05.207500+0000","last_undegraded":"2024-09-21T23:16:05.207500+0000","last_fullsized":"2024-09-21T23:16:05.207500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898304,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2710,"num_read_kb":24295,"num_write":114,"num_write_kb":2601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"63'483","reported_seq":1788,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.528567+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:16:04.528567+0000","last_peered":"2024-09-21T23:16:04.528567+0000","last_clean":"2024-09-21T23:16:04.528567+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:16:04.528567+0000","last_undegraded":"2024-09-21T23:16:04.528567+0000","last_fullsized":"2024-09-21T23:16:04.528567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:32.206028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260319,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":3003,"num_read_kb":11616,"num_write":727,"num_write_kb":1391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.163843+0000","last_change":"2024-09-21T23:15:06.350164+0000","last_active":"2024-09-21T23:16:04.163843+0000","last_peered":"2024-09-21T23:16:04.163843+0000","last_clean":"2024-09-21T23:16:04.163843+0000","last_became_active":"2024-09-21T23:15:06.349271+0000","last_became_peered":"2024-09-21T23:15:06.349271+0000","last_unstale":"2024-09-21T23:16:04.163843+0000","last_undegraded":"2024-09-21T23:16:04.163843+0000","last_fullsized":"2024-09-21T23:16:04.163843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:56.638782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"63'157","reported_seq":656,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.208419+0000","last_change":"2024-09-21T23:16:05.207857+0000","last_active":"2024-09-21T23:16:05.207857+0000","last_peered":"2024-09-21T23:16:04.750340+0000","last_clean":"2024-09-21T23:16:01.924947+0000","last_became_active":"2024-09-21T23:16:04.199145+0000","last_became_peered":"2024-09-21T23:16:04.199145+0000","last_unstale":"2024-09-21T23:16:05.208419+0000","last_undegraded":"2024-09-21T23:16:05.208419+0000","last_fullsized":"2024-09-21T23:16:05.208419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":20174888,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":355,"num_read_kb":2331,"num_write":154,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.189267+0000","last_change":"2024-09-21T23:16:04.189267+0000","last_active":"2024-09-21T23:16:04.189267+0000","last_peered":"2024-09-21T23:16:04.189267+0000","last_clean":"2024-09-21T23:16:04.189267+0000","last_became_active":"2024-09-21T23:16:04.189021+0000","last_became_peered":"2024-09-21T23:16:04.189021+0000","last_unstale":"2024-09-21T23:16:04.189267+0000","last_undegraded":"2024-09-21T23:16:04.189267+0000","last_fullsized":"2024-09-21T23:16:04.189267+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:19.263980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"58'1465","reported_seq":4149,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.162812+0000","last_change":"2024-09-21T23:15:35.969593+0000","last_active":"2024-09-21T23:16:04.162812+0000","last_peered":"2024-09-21T23:16:04.162812+0000","last_clean":"2024-09-21T23:16:04.162812+0000","last_became_active":"2024-09-21T23:15:35.967156+0000","last_became_peered":"2024-09-21T23:15:35.967156+0000","last_unstale":"2024-09-21T23:16:04.162812+0000","last_undegraded":"2024-09-21T23:16:04.162812+0000","last_fullsized":"2024-09-21T23:16:04.162812+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1465,"log_dups_size":0,"ondisk_log_size":1465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:27.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":7646122,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":9741,"num_read_kb":28640,"num_write":2644,"num_write_kb":5251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":247168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.207178+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:16:05.207178+0000","last_peered":"2024-09-21T23:16:05.207178+0000","last_clean":"2024-09-21T23:16:05.207178+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:16:05.207178+0000","last_undegraded":"2024-09-21T23:16:05.207178+0000","last_fullsized":"2024-09-21T23:16:05.207178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"59'116","reported_seq":741,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.527095+0000","last_change":"2024-09-21T23:16:04.314544+0000","last_active":"2024-09-21T23:16:04.527095+0000","last_peered":"2024-09-21T23:16:04.527095+0000","last_clean":"2024-09-21T23:16:01.928682+0000","last_became_active":"2024-09-21T23:16:04.198318+0000","last_became_peered":"2024-09-21T23:16:04.198318+0000","last_unstale":"2024-09-21T23:16:04.527095+0000","last_undegraded":"2024-09-21T23:16:04.527095+0000","last_fullsized":"2024-09-21T23:16:04.161145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":3896832,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":985,"num_read_kb":1563,"num_write":99,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":52736,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":11}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.206345+0000","last_change":"2024-09-21T23:16:04.198942+0000","last_active":"2024-09-21T23:16:05.206345+0000","last_peered":"2024-09-21T23:16:05.206345+0000","last_clean":"2024-09-21T23:16:05.206345+0000","last_became_active":"2024-09-21T23:16:04.198333+0000","last_became_peered":"2024-09-21T23:16:04.198333+0000","last_unstale":"2024-09-21T23:16:05.206345+0000","last_undegraded":"2024-09-21T23:16:05.206345+0000","last_fullsized":"2024-09-21T23:16:05.206345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:59.745128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"59'280","reported_seq":2299,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.513552+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:16:04.513552+0000","last_peered":"2024-09-21T23:16:04.513552+0000","last_clean":"2024-09-21T23:16:04.513552+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:16:04.513552+0000","last_undegraded":"2024-09-21T23:16:04.513552+0000","last_fullsized":"2024-09-21T23:16:04.513552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:31.015952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":9773854,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":3265,"num_read_kb":14249,"num_write":328,"num_write_kb":1511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'188","reported_seq":1803,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.199908+0000","last_change":"2024-09-21T23:16:04.200078+0000","last_active":"2024-09-21T23:16:04.199908+0000","last_peered":"2024-09-21T23:16:04.199908+0000","last_clean":"2024-09-21T23:15:10.359509+0000","last_became_active":"2024-09-21T23:16:04.199385+0000","last_became_peered":"2024-09-21T23:16:04.199385+0000","last_unstale":"2024-09-21T23:16:04.199908+0000","last_undegraded":"2024-09-21T23:16:04.165390+0000","last_fullsized":"2024-09-21T23:16:04.164568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":7304734,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":1643,"num_read_kb":10737,"num_write":197,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":13}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.205303+0000","last_change":"2024-09-21T23:16:04.205303+0000","last_active":"2024-09-21T23:16:04.205303+0000","last_peered":"2024-09-21T23:16:04.205303+0000","last_clean":"2024-09-21T23:16:04.205303+0000","last_became_active":"2024-09-21T23:16:04.181322+0000","last_became_peered":"2024-09-21T23:16:04.181322+0000","last_unstale":"2024-09-21T23:16:04.205303+0000","last_undegraded":"2024-09-21T23:16:04.205303+0000","last_fullsized":"2024-09-21T23:16:04.205303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:04.294981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'163","reported_seq":1840,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.512692+0000","last_change":"2024-09-21T23:15:35.969320+0000","last_active":"2024-09-21T23:16:04.512692+0000","last_peered":"2024-09-21T23:16:04.512692+0000","last_clean":"2024-09-21T23:16:04.512692+0000","last_became_active":"2024-09-21T23:15:35.968596+0000","last_became_peered":"2024-09-21T23:15:35.968596+0000","last_unstale":"2024-09-21T23:16:04.512692+0000","last_undegraded":"2024-09-21T23:16:04.512692+0000","last_fullsized":"2024-09-21T23:16:04.512692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:28:40.180849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":3327604,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1552,"num_read_kb":6767,"num_write":158,"num_write_kb":681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":29200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.197581+0000","last_change":"2024-09-21T23:16:04.197581+0000","last_active":"2024-09-21T23:16:04.197581+0000","last_peered":"2024-09-21T23:16:04.197581+0000","last_clean":"2024-09-21T23:16:04.197581+0000","last_became_active":"2024-09-21T23:16:04.196985+0000","last_became_peered":"2024-09-21T23:16:04.196985+0000","last_unstale":"2024-09-21T23:16:04.197581+0000","last_undegraded":"2024-09-21T23:16:04.197581+0000","last_fullsized":"2024-09-21T23:16:04.197581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:24:15.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'193","reported_seq":1352,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.207147+0000","last_change":"2024-09-21T23:16:05.207147+0000","last_active":"2024-09-21T23:16:03.161294+0000","last_peered":"2024-09-21T23:16:01.150966+0000","last_clean":"2024-09-21T23:15:08.563705+0000","last_became_active":"2024-09-21T23:15:11.742147+0000","last_became_peered":"2024-09-21T23:15:11.742147+0000","last_unstale":"2024-09-21T23:16:05.207147+0000","last_undegraded":"2024-09-21T23:16:05.207147+0000","last_fullsized":"2024-09-21T23:16:05.207147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":3952676,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":1107,"num_read_kb":6575,"num_write":197,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.201232+0000","last_change":"2024-09-21T23:16:04.207675+0000","last_active":"2024-09-21T23:16:05.201232+0000","last_peered":"2024-09-21T23:16:05.201232+0000","last_clean":"2024-09-21T23:16:05.201232+0000","last_became_active":"2024-09-21T23:16:04.206739+0000","last_became_peered":"2024-09-21T23:16:04.206739+0000","last_unstale":"2024-09-21T23:16:05.201232+0000","last_undegraded":"2024-09-21T23:16:05.201232+0000","last_fullsized":"2024-09-21T23:16:05.201232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:12.680088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"63'164","reported_seq":598,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.755609+0000","last_change":"2024-09-21T23:15:33.016643+0000","last_active":"2024-09-21T23:16:04.755609+0000","last_peered":"2024-09-21T23:16:04.755609+0000","last_clean":"2024-09-21T23:16:04.755609+0000","last_became_active":"2024-09-21T23:15:32.955127+0000","last_became_peered":"2024-09-21T23:15:32.955127+0000","last_unstale":"2024-09-21T23:16:04.755609+0000","last_undegraded":"2024-09-21T23:16:04.755609+0000","last_fullsized":"2024-09-21T23:16:04.755609+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:50.308037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":13215350,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":387,"num_read_kb":3830,"num_write":169,"num_write_kb":1817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":10301,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.155775+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:16:04.155775+0000","last_peered":"2024-09-21T23:16:04.155775+0000","last_clean":"2024-09-21T23:16:04.155775+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:16:04.155775+0000","last_undegraded":"2024-09-21T23:16:04.155775+0000","last_fullsized":"2024-09-21T23:16:04.155775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.207669+0000","last_change":"2024-09-21T23:16:04.207237+0000","last_active":"2024-09-21T23:16:05.207669+0000","last_peered":"2024-09-21T23:16:05.207669+0000","last_clean":"2024-09-21T23:16:05.207669+0000","last_became_active":"2024-09-21T23:16:04.206117+0000","last_became_peered":"2024-09-21T23:16:04.206117+0000","last_unstale":"2024-09-21T23:16:05.207669+0000","last_undegraded":"2024-09-21T23:16:05.207669+0000","last_fullsized":"2024-09-21T23:16:05.207669+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:26:34.280570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"59'508","reported_seq":9814,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.527080+0000","last_change":"2024-09-21T23:15:35.968122+0000","last_active":"2024-09-21T23:16:04.527080+0000","last_peered":"2024-09-21T23:16:04.527080+0000","last_clean":"2024-09-21T23:16:04.527080+0000","last_became_active":"2024-09-21T23:15:35.967154+0000","last_became_peered":"2024-09-21T23:15:35.967154+0000","last_unstale":"2024-09-21T23:16:04.527080+0000","last_undegraded":"2024-09-21T23:16:04.527080+0000","last_fullsized":"2024-09-21T23:16:04.527080+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:41.598133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":3641872,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":10525,"num_read_kb":14824,"num_write":300,"num_write_kb":1356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":15888,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"59'293","reported_seq":2856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.515765+0000","last_change":"2024-09-21T23:15:03.325366+0000","last_active":"2024-09-21T23:16:04.515765+0000","last_peered":"2024-09-21T23:16:04.515765+0000","last_clean":"2024-09-21T23:16:04.515765+0000","last_became_active":"2024-09-21T23:15:01.329082+0000","last_became_peered":"2024-09-21T23:15:01.329082+0000","last_unstale":"2024-09-21T23:16:04.515765+0000","last_undegraded":"2024-09-21T23:16:04.515765+0000","last_fullsized":"2024-09-21T23:16:04.515765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:46.526439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":5206528,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":4289,"num_read_kb":7869,"num_write":214,"num_write_kb":865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"59'127","reported_seq":442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.523750+0000","last_change":"2024-09-21T23:15:33.025560+0000","last_active":"2024-09-21T23:16:04.523750+0000","last_peered":"2024-09-21T23:16:04.523750+0000","last_clean":"2024-09-21T23:16:04.523750+0000","last_became_active":"2024-09-21T23:15:32.949370+0000","last_became_peered":"2024-09-21T23:15:32.949370+0000","last_unstale":"2024-09-21T23:16:04.523750+0000","last_undegraded":"2024-09-21T23:16:04.523750+0000","last_fullsized":"2024-09-21T23:16:04.523750+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:22.683465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988624,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":195,"num_read_kb":1010,"num_write":124,"num_write_kb":643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":8192,"num_keys_recovered":3,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"63'314","reported_seq":3708,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.657118+0000","last_change":"2024-09-21T23:16:04.206606+0000","last_active":"2024-09-21T23:16:04.657118+0000","last_peered":"2024-09-21T23:16:04.657118+0000","last_clean":"2024-09-21T23:16:01.975089+0000","last_became_active":"2024-09-21T23:16:04.205124+0000","last_became_peered":"2024-09-21T23:16:04.205124+0000","last_unstale":"2024-09-21T23:16:04.657118+0000","last_undegraded":"2024-09-21T23:16:04.165963+0000","last_fullsized":"2024-09-21T23:16:04.162982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":3923487,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":5208,"num_read_kb":13998,"num_write":220,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":94208,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[4],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,4","objects":17},{"shards":"4","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"59'202","reported_seq":667,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.571875+0000","last_change":"2024-09-21T23:16:04.572590+0000","last_active":"2024-09-21T23:16:04.571875+0000","last_peered":"2024-09-21T23:16:04.571875+0000","last_clean":"2024-09-21T23:15:10.367359+0000","last_became_active":"2024-09-21T23:16:04.198934+0000","last_became_peered":"2024-09-21T23:16:04.198934+0000","last_unstale":"2024-09-21T23:16:04.571875+0000","last_undegraded":"2024-09-21T23:16:04.156963+0000","last_fullsized":"2024-09-21T23:16:04.156016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":5001773,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":336,"num_read_kb":993,"num_write":210,"num_write_kb":1082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"5","objects":13}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'182","reported_seq":3236,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.513061+0000","last_change":"2024-09-21T23:15:06.357124+0000","last_active":"2024-09-21T23:16:04.513061+0000","last_peered":"2024-09-21T23:16:04.513061+0000","last_clean":"2024-09-21T23:16:04.513061+0000","last_became_active":"2024-09-21T23:15:06.356745+0000","last_became_peered":"2024-09-21T23:15:06.356745+0000","last_unstale":"2024-09-21T23:16:04.513061+0000","last_undegraded":"2024-09-21T23:16:04.513061+0000","last_fullsized":"2024-09-21T23:16:04.513061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:15.067781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5444218,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":3235,"num_read_kb":23473,"num_write":217,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"58'154","reported_seq":1881,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.155909+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:16:04.155909+0000","last_peered":"2024-09-21T23:16:04.155909+0000","last_clean":"2024-09-21T23:16:04.155909+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:16:04.155909+0000","last_undegraded":"2024-09-21T23:16:04.155909+0000","last_fullsized":"2024-09-21T23:16:04.155909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:08.946701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":4626432,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":2526,"num_read_kb":7546,"num_write":126,"num_write_kb":693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"58'138","reported_seq":2027,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.383573+0000","last_change":"2024-09-21T23:16:04.384089+0000","last_active":"2024-09-21T23:16:04.383573+0000","last_peered":"2024-09-21T23:16:04.383573+0000","last_clean":"2024-09-21T23:15:10.367429+0000","last_became_active":"2024-09-21T23:16:04.196883+0000","last_became_peered":"2024-09-21T23:16:04.196883+0000","last_unstale":"2024-09-21T23:16:04.383573+0000","last_undegraded":"2024-09-21T23:16:04.383573+0000","last_fullsized":"2024-09-21T23:16:04.156010+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":5008400,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":2053,"num_read_kb":6536,"num_write":117,"num_write_kb":494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"5","objects":9}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"59'174","reported_seq":1553,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.512598+0000","last_change":"2024-09-21T23:16:04.206447+0000","last_active":"2024-09-21T23:16:04.512598+0000","last_peered":"2024-09-21T23:16:04.512598+0000","last_clean":"2024-09-21T23:15:10.367211+0000","last_became_active":"2024-09-21T23:16:04.206033+0000","last_became_peered":"2024-09-21T23:16:04.206033+0000","last_unstale":"2024-09-21T23:16:04.512598+0000","last_undegraded":"2024-09-21T23:16:04.156677+0000","last_fullsized":"2024-09-21T23:16:04.155552+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":5904896,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1279,"num_read_kb":9175,"num_write":168,"num_write_kb":1018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":14}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"59'325","reported_seq":1798,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.516624+0000","last_change":"2024-09-21T23:16:04.444160+0000","last_active":"2024-09-21T23:16:04.516624+0000","last_peered":"2024-09-21T23:16:04.516624+0000","last_clean":"2024-09-21T23:15:10.359214+0000","last_became_active":"2024-09-21T23:16:04.195755+0000","last_became_peered":"2024-09-21T23:16:04.195755+0000","last_unstale":"2024-09-21T23:16:04.516624+0000","last_undegraded":"2024-09-21T23:16:04.516624+0000","last_fullsized":"2024-09-21T23:16:04.162831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2856485,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":2459,"num_read_kb":5407,"num_write":489,"num_write_kb":933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":["4","0"],"object_location_counts":[{"shards":"4","objects":14}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"63'337","reported_seq":4648,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.202268+0000","last_change":"2024-09-21T23:16:05.200483+0000","last_active":"2024-09-21T23:16:05.200483+0000","last_peered":"2024-09-21T23:16:04.675100+0000","last_clean":"2024-09-21T23:16:01.157282+0000","last_became_active":"2024-09-21T23:16:04.206272+0000","last_became_peered":"2024-09-21T23:16:04.206272+0000","last_unstale":"2024-09-21T23:16:05.202268+0000","last_undegraded":"2024-09-21T23:16:05.202268+0000","last_fullsized":"2024-09-21T23:16:05.202268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":10283580,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":6198,"num_read_kb":14619,"num_write":239,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":154112,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'161","reported_seq":7605,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.524795+0000","last_change":"2024-09-21T23:15:06.345498+0000","last_active":"2024-09-21T23:16:04.524795+0000","last_peered":"2024-09-21T23:16:04.524795+0000","last_clean":"2024-09-21T23:16:04.524795+0000","last_became_active":"2024-09-21T23:15:06.344491+0000","last_became_peered":"2024-09-21T23:15:06.344491+0000","last_unstale":"2024-09-21T23:16:04.524795+0000","last_undegraded":"2024-09-21T23:16:04.524795+0000","last_fullsized":"2024-09-21T23:16:04.524795+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:30:45.230225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7422480,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":7374,"num_read_kb":16230,"num_write":152,"num_write_kb":763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":233789977,"num_objects":447,"num_object_clones":115,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":143,"num_read":89812,"num_read_kb":321033,"num_write":9618,"num_write_kb":50306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":488,"num_bytes_recovered":1159483,"num_keys_recovered":60,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43405312,"data_stored":52380869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":8262,"ondisk_log_size":8262,"up":64,"acting":56,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52084,"kb_used_data":12572,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319756,"statfs":{"total":96636764160,"available":96583430144,"internally_reserved":0,"allocated":12873728,"data_stored":16126365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":15672,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":16048128,"data_stored":19027088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607572,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607572,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61932,"kb_used_data":8532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309908,"statfs":{"total":96636764160,"available":96573345792,"internally_reserved":0,"allocated":8736768,"data_stored":9395283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607573,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005058,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":7560,"kb_used_omap":5,"kb_used_meta":32314,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":7741440,"data_stored":8176066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":33089829},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6410240,"data_stored":7286773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1794048,"data_stored":1745027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":8516971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15503360,"data_stored":18895084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12259328,"data_stored":15936998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:16:09.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T23:16:09.140 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:09.140+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:09.212 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:09.212+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:09.225 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:09.225 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:09.227 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:16:09.241 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:09.312 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:09.312+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:09.362 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:09.362 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:09.377 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:09.410 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone15 from 0xdfad400 (overlap 0xae7a600) to 0x573d200 2024-09-21T23:16:09.426 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:09.419+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:09.513 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:09.513 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:09.513 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:09.507+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:09.525 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:09.525 INFO:teuthology.orchestra.run.smithi053.stdout:{"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 128/902 objects degraded (14.191%), 4 pgs degraded","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":97,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":68,"num_osds":6,"num_up_osds":6,"osd_up_since":1726960562,"num_in_osds":4,"osd_in_since":1726960531,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":26},{"state_name":"peering","count":7},{"state_name":"active+undersized+remapped","count":4},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":451,"data_bytes":234380364,"bytes_used":218378240,"bytes_avail":386328678400,"bytes_total":386547056640,"inactive_pgs_ratio":0.17073170840740204,"degraded_objects":128,"degraded_total":902,"degraded_ratio":0.14190687361419069,"recovering_objects_per_sec":53,"recovering_bytes_per_sec":148729,"recovering_keys_per_sec":8,"num_objects_recovered":213,"num_bytes_recovered":595116,"num_keys_recovered":35,"read_bytes_sec":1284694,"write_bytes_sec":83172,"read_op_per_sec":522,"write_op_per_sec":31},"fsmap":{"epoch":1,"btime":"2024-09-21T23:14:32:027953+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-21T23:15:46.857609+0000","services":{}},"progress_events":{}} 2024-09-21T23:16:09.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:09.551 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T23:16:09.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:16:09.614 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:09.611+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:09.696 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:09.696 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:09.711 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:09.715 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:09.712+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:09.816 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:09.816+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:09.884 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:09.885 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:09.901 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:09.917 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:09.916+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:10.017 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:10.016+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:10.026 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:10.026 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:10.043 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:16:10.102 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-21T23:16:09.228147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":234380364,"num_objects":451,"num_object_clones":115,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":143,"num_read":89928,"num_read_kb":321133,"num_write":9779,"num_write_kb":52724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":1749870,"num_keys_recovered":60,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8382,"ondisk_log_size":8382,"up":82,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":213260,"kb_used_data":44336,"kb_used_omap":10,"kb_used_meta":168821,"kb_avail":377274100,"statfs":{"total":386547056640,"available":386328678400,"internally_reserved":0,"allocated":45400064,"data_stored":52724802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":172873332},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-33,"num_objects_misplaced":-2,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-112,"num_read_kb":-471,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-120,"num_bytes_recovered":-3603582,"num_keys_recovered":-12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.037627"},"pg_stats":[{"pgid":"3.1f","version":"63'175","reported_seq":392,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.201320+0000","last_change":"2024-09-21T23:16:05.201211+0000","last_active":"2024-09-21T23:16:05.201211+0000","last_peered":"2024-09-21T23:16:04.648784+0000","last_clean":"2024-09-21T23:15:08.357378+0000","last_became_active":"2024-09-21T23:16:04.202564+0000","last_became_peered":"2024-09-21T23:16:04.202564+0000","last_unstale":"2024-09-21T23:16:05.201320+0000","last_undegraded":"2024-09-21T23:16:05.201320+0000","last_fullsized":"2024-09-21T23:16:05.201320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":6582836,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":425,"num_read_kb":473,"num_write":222,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":14388,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"59'181","reported_seq":996,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.527236+0000","last_change":"2024-09-21T23:15:35.968961+0000","last_active":"2024-09-21T23:16:04.527236+0000","last_peered":"2024-09-21T23:16:04.527236+0000","last_clean":"2024-09-21T23:16:04.527236+0000","last_became_active":"2024-09-21T23:15:35.968136+0000","last_became_peered":"2024-09-21T23:15:35.968136+0000","last_unstale":"2024-09-21T23:16:04.527236+0000","last_undegraded":"2024-09-21T23:16:04.527236+0000","last_fullsized":"2024-09-21T23:16:04.527236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:59.548525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":8337956,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":698,"num_read_kb":9271,"num_write":173,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"59'81","reported_seq":1204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.525067+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:16:04.525067+0000","last_peered":"2024-09-21T23:16:04.525067+0000","last_clean":"2024-09-21T23:16:04.525067+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:16:04.525067+0000","last_undegraded":"2024-09-21T23:16:04.525067+0000","last_fullsized":"2024-09-21T23:16:04.525067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:47:53.671106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":4360704,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1031,"num_read_kb":5876,"num_write":82,"num_write_kb":1980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'217","reported_seq":2391,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.766882+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:16:04.766882+0000","last_peered":"2024-09-21T23:16:04.766882+0000","last_clean":"2024-09-21T23:16:04.766882+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:16:04.766882+0000","last_undegraded":"2024-09-21T23:16:04.766882+0000","last_fullsized":"2024-09-21T23:16:04.766882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:01:28.334770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":10842176,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":2295,"num_read_kb":16616,"num_write":202,"num_write_kb":5067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"58'231","reported_seq":2023,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.206485+0000","last_change":"2024-09-21T23:16:05.206485+0000","last_active":"2024-09-21T23:16:03.163194+0000","last_peered":"2024-09-21T23:16:01.151190+0000","last_clean":"2024-09-21T23:15:08.357275+0000","last_became_active":"2024-09-21T23:15:11.742173+0000","last_became_peered":"2024-09-21T23:15:11.742173+0000","last_unstale":"2024-09-21T23:16:05.206485+0000","last_undegraded":"2024-09-21T23:16:05.206485+0000","last_fullsized":"2024-09-21T23:16:05.206485+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":5812766,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":2685,"num_read_kb":11595,"num_write":252,"num_write_kb":874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"63'280","reported_seq":5173,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.740820+0000","last_change":"2024-09-21T23:15:35.969265+0000","last_active":"2024-09-21T23:16:04.740820+0000","last_peered":"2024-09-21T23:16:04.740820+0000","last_clean":"2024-09-21T23:16:04.740820+0000","last_became_active":"2024-09-21T23:15:35.967793+0000","last_became_peered":"2024-09-21T23:15:35.967793+0000","last_unstale":"2024-09-21T23:16:04.740820+0000","last_undegraded":"2024-09-21T23:16:04.740820+0000","last_fullsized":"2024-09-21T23:16:04.740820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:02:28.996499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":4649507,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":6577,"num_read_kb":16480,"num_write":207,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":39443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"58'114","reported_seq":1120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.164122+0000","last_change":"2024-09-21T23:15:32.983007+0000","last_active":"2024-09-21T23:16:04.164122+0000","last_peered":"2024-09-21T23:16:04.164122+0000","last_clean":"2024-09-21T23:16:04.164122+0000","last_became_active":"2024-09-21T23:15:32.948191+0000","last_became_peered":"2024-09-21T23:15:32.948191+0000","last_unstale":"2024-09-21T23:16:04.164122+0000","last_undegraded":"2024-09-21T23:16:04.164122+0000","last_fullsized":"2024-09-21T23:16:04.164122+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:34:04.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":4866048,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":876,"num_read_kb":6282,"num_write":113,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"63'321","reported_seq":2016,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.879668+0000","last_change":"2024-09-21T23:16:04.865800+0000","last_active":"2024-09-21T23:16:04.879668+0000","last_peered":"2024-09-21T23:16:04.879668+0000","last_clean":"2024-09-21T23:15:10.369468+0000","last_became_active":"2024-09-21T23:16:04.210375+0000","last_became_peered":"2024-09-21T23:16:04.210375+0000","last_unstale":"2024-09-21T23:16:04.879668+0000","last_undegraded":"2024-09-21T23:16:04.167066+0000","last_fullsized":"2024-09-21T23:16:04.160794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":10488892,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":2675,"num_read_kb":13513,"num_write":481,"num_write_kb":2839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"63'149","reported_seq":490,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.201797+0000","last_change":"2024-09-21T23:16:05.201645+0000","last_active":"2024-09-21T23:16:05.201645+0000","last_peered":"2024-09-21T23:16:04.750843+0000","last_clean":"2024-09-21T23:15:09.361363+0000","last_became_active":"2024-09-21T23:16:04.202539+0000","last_became_peered":"2024-09-21T23:16:04.202539+0000","last_unstale":"2024-09-21T23:16:05.201797+0000","last_undegraded":"2024-09-21T23:16:05.201797+0000","last_fullsized":"2024-09-21T23:16:05.201797+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":11392064,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":226,"num_read_kb":1884,"num_write":146,"num_write_kb":681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":25664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"59'274","reported_seq":1560,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.510157+0000","last_change":"2024-09-21T23:15:35.965248+0000","last_active":"2024-09-21T23:16:04.510157+0000","last_peered":"2024-09-21T23:16:04.510157+0000","last_clean":"2024-09-21T23:16:04.510157+0000","last_became_active":"2024-09-21T23:15:35.963512+0000","last_became_peered":"2024-09-21T23:15:35.963512+0000","last_unstale":"2024-09-21T23:16:04.510157+0000","last_undegraded":"2024-09-21T23:16:04.510157+0000","last_fullsized":"2024-09-21T23:16:04.510157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:44:51.065083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":15697570,"num_objects":25,"num_object_clones":9,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1899,"num_read_kb":6760,"num_write":377,"num_write_kb":2104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":94370,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"58'115","reported_seq":2909,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.207500+0000","last_change":"2024-09-21T23:16:05.207500+0000","last_active":"2024-09-21T23:16:03.162957+0000","last_peered":"2024-09-21T23:16:01.151932+0000","last_clean":"2024-09-21T23:15:08.790498+0000","last_became_active":"2024-09-21T23:15:11.743088+0000","last_became_peered":"2024-09-21T23:15:11.743088+0000","last_unstale":"2024-09-21T23:16:05.207500+0000","last_undegraded":"2024-09-21T23:16:05.207500+0000","last_fullsized":"2024-09-21T23:16:05.207500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898304,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2710,"num_read_kb":24295,"num_write":114,"num_write_kb":2601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"63'483","reported_seq":1788,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.528567+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:16:04.528567+0000","last_peered":"2024-09-21T23:16:04.528567+0000","last_clean":"2024-09-21T23:16:04.528567+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:16:04.528567+0000","last_undegraded":"2024-09-21T23:16:04.528567+0000","last_fullsized":"2024-09-21T23:16:04.528567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:32.206028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7260319,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":3003,"num_read_kb":11616,"num_write":727,"num_write_kb":1391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.163843+0000","last_change":"2024-09-21T23:15:06.350164+0000","last_active":"2024-09-21T23:16:04.163843+0000","last_peered":"2024-09-21T23:16:04.163843+0000","last_clean":"2024-09-21T23:16:04.163843+0000","last_became_active":"2024-09-21T23:15:06.349271+0000","last_became_peered":"2024-09-21T23:15:06.349271+0000","last_unstale":"2024-09-21T23:16:04.163843+0000","last_undegraded":"2024-09-21T23:16:04.163843+0000","last_fullsized":"2024-09-21T23:16:04.163843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:56.638782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"63'157","reported_seq":656,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.208419+0000","last_change":"2024-09-21T23:16:05.207857+0000","last_active":"2024-09-21T23:16:05.207857+0000","last_peered":"2024-09-21T23:16:04.750340+0000","last_clean":"2024-09-21T23:16:01.924947+0000","last_became_active":"2024-09-21T23:16:04.199145+0000","last_became_peered":"2024-09-21T23:16:04.199145+0000","last_unstale":"2024-09-21T23:16:05.208419+0000","last_undegraded":"2024-09-21T23:16:05.208419+0000","last_fullsized":"2024-09-21T23:16:05.208419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":20174888,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":355,"num_read_kb":2331,"num_write":154,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.189267+0000","last_change":"2024-09-21T23:16:04.189267+0000","last_active":"2024-09-21T23:16:04.189267+0000","last_peered":"2024-09-21T23:16:04.189267+0000","last_clean":"2024-09-21T23:16:04.189267+0000","last_became_active":"2024-09-21T23:16:04.189021+0000","last_became_peered":"2024-09-21T23:16:04.189021+0000","last_unstale":"2024-09-21T23:16:04.189267+0000","last_undegraded":"2024-09-21T23:16:04.189267+0000","last_fullsized":"2024-09-21T23:16:04.189267+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:19.263980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"58'1465","reported_seq":4149,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.162812+0000","last_change":"2024-09-21T23:15:35.969593+0000","last_active":"2024-09-21T23:16:04.162812+0000","last_peered":"2024-09-21T23:16:04.162812+0000","last_clean":"2024-09-21T23:16:04.162812+0000","last_became_active":"2024-09-21T23:15:35.967156+0000","last_became_peered":"2024-09-21T23:15:35.967156+0000","last_unstale":"2024-09-21T23:16:04.162812+0000","last_undegraded":"2024-09-21T23:16:04.162812+0000","last_fullsized":"2024-09-21T23:16:04.162812+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1465,"log_dups_size":0,"ondisk_log_size":1465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:27.893737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":7646122,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":9741,"num_read_kb":28640,"num_write":2644,"num_write_kb":5251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":247168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.207178+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:16:05.207178+0000","last_peered":"2024-09-21T23:16:05.207178+0000","last_clean":"2024-09-21T23:16:05.207178+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:16:05.207178+0000","last_undegraded":"2024-09-21T23:16:05.207178+0000","last_fullsized":"2024-09-21T23:16:05.207178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"59'116","reported_seq":741,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.527095+0000","last_change":"2024-09-21T23:16:04.314544+0000","last_active":"2024-09-21T23:16:04.527095+0000","last_peered":"2024-09-21T23:16:04.527095+0000","last_clean":"2024-09-21T23:16:01.928682+0000","last_became_active":"2024-09-21T23:16:04.198318+0000","last_became_peered":"2024-09-21T23:16:04.198318+0000","last_unstale":"2024-09-21T23:16:04.527095+0000","last_undegraded":"2024-09-21T23:16:04.527095+0000","last_fullsized":"2024-09-21T23:16:04.161145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":3896832,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":985,"num_read_kb":1563,"num_write":99,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":52736,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":11}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.206345+0000","last_change":"2024-09-21T23:16:04.198942+0000","last_active":"2024-09-21T23:16:05.206345+0000","last_peered":"2024-09-21T23:16:05.206345+0000","last_clean":"2024-09-21T23:16:05.206345+0000","last_became_active":"2024-09-21T23:16:04.198333+0000","last_became_peered":"2024-09-21T23:16:04.198333+0000","last_unstale":"2024-09-21T23:16:05.206345+0000","last_undegraded":"2024-09-21T23:16:05.206345+0000","last_fullsized":"2024-09-21T23:16:05.206345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:59.745128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"59'280","reported_seq":2299,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.513552+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:16:04.513552+0000","last_peered":"2024-09-21T23:16:04.513552+0000","last_clean":"2024-09-21T23:16:04.513552+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:16:04.513552+0000","last_undegraded":"2024-09-21T23:16:04.513552+0000","last_fullsized":"2024-09-21T23:16:04.513552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:31.015952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":9773854,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":3265,"num_read_kb":14249,"num_write":328,"num_write_kb":1511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'188","reported_seq":1803,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.199908+0000","last_change":"2024-09-21T23:16:04.200078+0000","last_active":"2024-09-21T23:16:04.199908+0000","last_peered":"2024-09-21T23:16:04.199908+0000","last_clean":"2024-09-21T23:15:10.359509+0000","last_became_active":"2024-09-21T23:16:04.199385+0000","last_became_peered":"2024-09-21T23:16:04.199385+0000","last_unstale":"2024-09-21T23:16:04.199908+0000","last_undegraded":"2024-09-21T23:16:04.165390+0000","last_fullsized":"2024-09-21T23:16:04.164568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":7304734,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":1643,"num_read_kb":10737,"num_write":197,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":13}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.205303+0000","last_change":"2024-09-21T23:16:04.205303+0000","last_active":"2024-09-21T23:16:04.205303+0000","last_peered":"2024-09-21T23:16:04.205303+0000","last_clean":"2024-09-21T23:16:04.205303+0000","last_became_active":"2024-09-21T23:16:04.181322+0000","last_became_peered":"2024-09-21T23:16:04.181322+0000","last_unstale":"2024-09-21T23:16:04.205303+0000","last_undegraded":"2024-09-21T23:16:04.205303+0000","last_fullsized":"2024-09-21T23:16:04.205303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:04.294981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'163","reported_seq":1840,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.512692+0000","last_change":"2024-09-21T23:15:35.969320+0000","last_active":"2024-09-21T23:16:04.512692+0000","last_peered":"2024-09-21T23:16:04.512692+0000","last_clean":"2024-09-21T23:16:04.512692+0000","last_became_active":"2024-09-21T23:15:35.968596+0000","last_became_peered":"2024-09-21T23:15:35.968596+0000","last_unstale":"2024-09-21T23:16:04.512692+0000","last_undegraded":"2024-09-21T23:16:04.512692+0000","last_fullsized":"2024-09-21T23:16:04.512692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:28:40.180849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":3327604,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1552,"num_read_kb":6767,"num_write":158,"num_write_kb":681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":29200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.197581+0000","last_change":"2024-09-21T23:16:04.197581+0000","last_active":"2024-09-21T23:16:04.197581+0000","last_peered":"2024-09-21T23:16:04.197581+0000","last_clean":"2024-09-21T23:16:04.197581+0000","last_became_active":"2024-09-21T23:16:04.196985+0000","last_became_peered":"2024-09-21T23:16:04.196985+0000","last_unstale":"2024-09-21T23:16:04.197581+0000","last_undegraded":"2024-09-21T23:16:04.197581+0000","last_fullsized":"2024-09-21T23:16:04.197581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:24:15.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'193","reported_seq":1352,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.207147+0000","last_change":"2024-09-21T23:16:05.207147+0000","last_active":"2024-09-21T23:16:03.161294+0000","last_peered":"2024-09-21T23:16:01.150966+0000","last_clean":"2024-09-21T23:15:08.563705+0000","last_became_active":"2024-09-21T23:15:11.742147+0000","last_became_peered":"2024-09-21T23:15:11.742147+0000","last_unstale":"2024-09-21T23:16:05.207147+0000","last_undegraded":"2024-09-21T23:16:05.207147+0000","last_fullsized":"2024-09-21T23:16:05.207147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":3952676,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":1107,"num_read_kb":6575,"num_write":197,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.201232+0000","last_change":"2024-09-21T23:16:04.207675+0000","last_active":"2024-09-21T23:16:05.201232+0000","last_peered":"2024-09-21T23:16:05.201232+0000","last_clean":"2024-09-21T23:16:05.201232+0000","last_became_active":"2024-09-21T23:16:04.206739+0000","last_became_peered":"2024-09-21T23:16:04.206739+0000","last_unstale":"2024-09-21T23:16:05.201232+0000","last_undegraded":"2024-09-21T23:16:05.201232+0000","last_fullsized":"2024-09-21T23:16:05.201232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:12.680088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"63'164","reported_seq":598,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.755609+0000","last_change":"2024-09-21T23:15:33.016643+0000","last_active":"2024-09-21T23:16:04.755609+0000","last_peered":"2024-09-21T23:16:04.755609+0000","last_clean":"2024-09-21T23:16:04.755609+0000","last_became_active":"2024-09-21T23:15:32.955127+0000","last_became_peered":"2024-09-21T23:15:32.955127+0000","last_unstale":"2024-09-21T23:16:04.755609+0000","last_undegraded":"2024-09-21T23:16:04.755609+0000","last_fullsized":"2024-09-21T23:16:04.755609+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:50.308037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":13215350,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":387,"num_read_kb":3830,"num_write":169,"num_write_kb":1817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":10301,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.155775+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:16:04.155775+0000","last_peered":"2024-09-21T23:16:04.155775+0000","last_clean":"2024-09-21T23:16:04.155775+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:16:04.155775+0000","last_undegraded":"2024-09-21T23:16:04.155775+0000","last_fullsized":"2024-09-21T23:16:04.155775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T23:16:05.207669+0000","last_change":"2024-09-21T23:16:04.207237+0000","last_active":"2024-09-21T23:16:05.207669+0000","last_peered":"2024-09-21T23:16:05.207669+0000","last_clean":"2024-09-21T23:16:05.207669+0000","last_became_active":"2024-09-21T23:16:04.206117+0000","last_became_peered":"2024-09-21T23:16:04.206117+0000","last_unstale":"2024-09-21T23:16:05.207669+0000","last_undegraded":"2024-09-21T23:16:05.207669+0000","last_fullsized":"2024-09-21T23:16:05.207669+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:26:34.280570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"59'508","reported_seq":9814,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.527080+0000","last_change":"2024-09-21T23:15:35.968122+0000","last_active":"2024-09-21T23:16:04.527080+0000","last_peered":"2024-09-21T23:16:04.527080+0000","last_clean":"2024-09-21T23:16:04.527080+0000","last_became_active":"2024-09-21T23:15:35.967154+0000","last_became_peered":"2024-09-21T23:15:35.967154+0000","last_unstale":"2024-09-21T23:16:04.527080+0000","last_undegraded":"2024-09-21T23:16:04.527080+0000","last_fullsized":"2024-09-21T23:16:04.527080+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:41.598133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":3641872,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":10525,"num_read_kb":14824,"num_write":300,"num_write_kb":1356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":15888,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"59'293","reported_seq":2856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.515765+0000","last_change":"2024-09-21T23:15:03.325366+0000","last_active":"2024-09-21T23:16:04.515765+0000","last_peered":"2024-09-21T23:16:04.515765+0000","last_clean":"2024-09-21T23:16:04.515765+0000","last_became_active":"2024-09-21T23:15:01.329082+0000","last_became_peered":"2024-09-21T23:15:01.329082+0000","last_unstale":"2024-09-21T23:16:04.515765+0000","last_undegraded":"2024-09-21T23:16:04.515765+0000","last_fullsized":"2024-09-21T23:16:04.515765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:46.526439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":5206528,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":4289,"num_read_kb":7869,"num_write":214,"num_write_kb":865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"59'127","reported_seq":442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.523750+0000","last_change":"2024-09-21T23:15:33.025560+0000","last_active":"2024-09-21T23:16:04.523750+0000","last_peered":"2024-09-21T23:16:04.523750+0000","last_clean":"2024-09-21T23:16:04.523750+0000","last_became_active":"2024-09-21T23:15:32.949370+0000","last_became_peered":"2024-09-21T23:15:32.949370+0000","last_unstale":"2024-09-21T23:16:04.523750+0000","last_undegraded":"2024-09-21T23:16:04.523750+0000","last_fullsized":"2024-09-21T23:16:04.523750+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:22.683465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988624,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":195,"num_read_kb":1010,"num_write":124,"num_write_kb":643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":8192,"num_keys_recovered":3,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"63'314","reported_seq":3708,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.657118+0000","last_change":"2024-09-21T23:16:04.206606+0000","last_active":"2024-09-21T23:16:04.657118+0000","last_peered":"2024-09-21T23:16:04.657118+0000","last_clean":"2024-09-21T23:16:01.975089+0000","last_became_active":"2024-09-21T23:16:04.205124+0000","last_became_peered":"2024-09-21T23:16:04.205124+0000","last_unstale":"2024-09-21T23:16:04.657118+0000","last_undegraded":"2024-09-21T23:16:04.165963+0000","last_fullsized":"2024-09-21T23:16:04.162982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":3923487,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":5208,"num_read_kb":13998,"num_write":220,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":94208,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[4],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,4","objects":17},{"shards":"4","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"59'202","reported_seq":667,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.571875+0000","last_change":"2024-09-21T23:16:04.572590+0000","last_active":"2024-09-21T23:16:04.571875+0000","last_peered":"2024-09-21T23:16:04.571875+0000","last_clean":"2024-09-21T23:15:10.367359+0000","last_became_active":"2024-09-21T23:16:04.198934+0000","last_became_peered":"2024-09-21T23:16:04.198934+0000","last_unstale":"2024-09-21T23:16:04.571875+0000","last_undegraded":"2024-09-21T23:16:04.156963+0000","last_fullsized":"2024-09-21T23:16:04.156016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":5001773,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":336,"num_read_kb":993,"num_write":210,"num_write_kb":1082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"5","objects":13}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'182","reported_seq":3236,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.513061+0000","last_change":"2024-09-21T23:15:06.357124+0000","last_active":"2024-09-21T23:16:04.513061+0000","last_peered":"2024-09-21T23:16:04.513061+0000","last_clean":"2024-09-21T23:16:04.513061+0000","last_became_active":"2024-09-21T23:15:06.356745+0000","last_became_peered":"2024-09-21T23:15:06.356745+0000","last_unstale":"2024-09-21T23:16:04.513061+0000","last_undegraded":"2024-09-21T23:16:04.513061+0000","last_fullsized":"2024-09-21T23:16:04.513061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:15.067781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5444218,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":3235,"num_read_kb":23473,"num_write":217,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"58'154","reported_seq":1881,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.155909+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:16:04.155909+0000","last_peered":"2024-09-21T23:16:04.155909+0000","last_clean":"2024-09-21T23:16:04.155909+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:16:04.155909+0000","last_undegraded":"2024-09-21T23:16:04.155909+0000","last_fullsized":"2024-09-21T23:16:04.155909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:08.946701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":4626432,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":2526,"num_read_kb":7546,"num_write":126,"num_write_kb":693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"58'138","reported_seq":2027,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.383573+0000","last_change":"2024-09-21T23:16:04.384089+0000","last_active":"2024-09-21T23:16:04.383573+0000","last_peered":"2024-09-21T23:16:04.383573+0000","last_clean":"2024-09-21T23:15:10.367429+0000","last_became_active":"2024-09-21T23:16:04.196883+0000","last_became_peered":"2024-09-21T23:16:04.196883+0000","last_unstale":"2024-09-21T23:16:04.383573+0000","last_undegraded":"2024-09-21T23:16:04.383573+0000","last_fullsized":"2024-09-21T23:16:04.156010+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":5008400,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":2053,"num_read_kb":6536,"num_write":117,"num_write_kb":494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"5","objects":9}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"59'174","reported_seq":1553,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-21T23:16:04.512598+0000","last_change":"2024-09-21T23:16:04.206447+0000","last_active":"2024-09-21T23:16:04.512598+0000","last_peered":"2024-09-21T23:16:04.512598+0000","last_clean":"2024-09-21T23:15:10.367211+0000","last_became_active":"2024-09-21T23:16:04.206033+0000","last_became_peered":"2024-09-21T23:16:04.206033+0000","last_unstale":"2024-09-21T23:16:04.512598+0000","last_undegraded":"2024-09-21T23:16:04.156677+0000","last_fullsized":"2024-09-21T23:16:04.155552+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":5904896,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1279,"num_read_kb":9175,"num_write":168,"num_write_kb":1018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":14}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"59'325","reported_seq":1798,"reported_epoch":63,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:04.516624+0000","last_change":"2024-09-21T23:16:04.444160+0000","last_active":"2024-09-21T23:16:04.516624+0000","last_peered":"2024-09-21T23:16:04.516624+0000","last_clean":"2024-09-21T23:15:10.359214+0000","last_became_active":"2024-09-21T23:16:04.195755+0000","last_became_peered":"2024-09-21T23:16:04.195755+0000","last_unstale":"2024-09-21T23:16:04.516624+0000","last_undegraded":"2024-09-21T23:16:04.516624+0000","last_fullsized":"2024-09-21T23:16:04.162831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2856485,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":2459,"num_read_kb":5407,"num_write":489,"num_write_kb":933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":["4","0"],"object_location_counts":[{"shards":"4","objects":14}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"63'337","reported_seq":4648,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-21T23:16:05.202268+0000","last_change":"2024-09-21T23:16:05.200483+0000","last_active":"2024-09-21T23:16:05.200483+0000","last_peered":"2024-09-21T23:16:04.675100+0000","last_clean":"2024-09-21T23:16:01.157282+0000","last_became_active":"2024-09-21T23:16:04.206272+0000","last_became_peered":"2024-09-21T23:16:04.206272+0000","last_unstale":"2024-09-21T23:16:05.202268+0000","last_undegraded":"2024-09-21T23:16:05.202268+0000","last_fullsized":"2024-09-21T23:16:05.202268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":10283580,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":6198,"num_read_kb":14619,"num_write":239,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":154112,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'161","reported_seq":7605,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T23:16:04.524795+0000","last_change":"2024-09-21T23:15:06.345498+0000","last_active":"2024-09-21T23:16:04.524795+0000","last_peered":"2024-09-21T23:16:04.524795+0000","last_clean":"2024-09-21T23:16:04.524795+0000","last_became_active":"2024-09-21T23:15:06.344491+0000","last_became_peered":"2024-09-21T23:15:06.344491+0000","last_unstale":"2024-09-21T23:16:04.524795+0000","last_undegraded":"2024-09-21T23:16:04.524795+0000","last_fullsized":"2024-09-21T23:16:04.524795+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:30:45.230225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7422480,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":7374,"num_read_kb":16230,"num_write":152,"num_write_kb":763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":233789977,"num_objects":447,"num_object_clones":115,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":143,"num_read":89812,"num_read_kb":321033,"num_write":9618,"num_write_kb":50306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":488,"num_bytes_recovered":1159483,"num_keys_recovered":60,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43405312,"data_stored":52380869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":8262,"ondisk_log_size":8262,"up":64,"acting":56,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52084,"kb_used_data":12572,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319756,"statfs":{"total":96636764160,"available":96583430144,"internally_reserved":0,"allocated":12873728,"data_stored":16126365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":15672,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":16048128,"data_stored":19027088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607572,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607572,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61932,"kb_used_data":8532,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309908,"statfs":{"total":96636764160,"available":96573345792,"internally_reserved":0,"allocated":8736768,"data_stored":9395283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607573,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005058,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":7560,"kb_used_omap":5,"kb_used_meta":32314,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":7741440,"data_stored":8176066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":33089829},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6410240,"data_stored":7286773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1794048,"data_stored":1745027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7434240,"data_stored":8516971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15503360,"data_stored":18895084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12259328,"data_stored":15936998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:16:10.144 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:10.144+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:10.163 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:10.163 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:10.178 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:10.220 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:10.220+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:10.306 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:10.306 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:10.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:10.321 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:10.315+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:10.422 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:10.415+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:10.486 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:10.486 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:10.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:10.552 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:10.552+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:10.623 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:10.619+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:10.633 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:10.634 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:10.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:10.663 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:10.724 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:10.719+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:10.807 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:10.807 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:10.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:10.824 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:10.819+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:10.828 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:10.846 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:10.846 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:10.863 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:10.864 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:10.884 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:10.884 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:10.904 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:10.904 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:10.924 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:10.925 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:10.919+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:10.937 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:10.937 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:10.937 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:10.937 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:10.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:11.026 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:11.019+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:11.111 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:11.112 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:11.125 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:11.127 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:11.123+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:11.227 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:11.223+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:11.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:16:11.259 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:421 3.2 3:440f700a:::rbd_object_map.10f16b870c42.0000000000000003:head [writefull 0~45] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 63.914202144999997, 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010238254, 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/398956569", 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 421 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345197+0000", 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345200+0000", 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0699999999999998e-06 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345215+0000", 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5273000000000001e-05 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345248+0000", 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2589000000000003e-05 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345268+0000", 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0446999999999999e-05 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345443+0000", 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017539899999999999 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.345457+0000", 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4053000000000001e-05 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351464+0000", 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060062029999999999 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.351802+0000", 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033875099999999998 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.352167+0000", 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036471399999999998 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.354295+0000", 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021280510000000002 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355221+0000", 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092538600000000005 2024-09-21T23:16:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355352+0000", 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013165900000000001 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:07.355435+0000", 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2658999999999996e-05 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4405.0:695 3.16 e27/25 3:6d83b35b:::rbd_object_map.10f16a447f06:head v 27'18, mlcod=27'14)", 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 63.146559783999997, 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046717140000000004, 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112839+0000", 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112841+0000", 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7999999999999999e-06 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112853+0000", 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2006e-05 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112877+0000", 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.112891+0000", 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4600000000000001e-05 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113404+0000", 2024-09-21T23:16:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00051235899999999999 2024-09-21T23:16:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.113437+0000", 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3543000000000001e-05 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117415+0000", 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039778130000000002 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:08.117511+0000", 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5798999999999995e-05 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6107 3.19 3:9befa1df:::rbd_data.10f1e443a6fa.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.767249702000001, 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007638494, 2024-09-21T23:16:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6107 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492149+0000", 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492151+0000", 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5179999999999999e-06 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492159+0000", 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-06 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492176+0000", 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7581999999999999e-05 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492189+0000", 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2548000000000001e-05 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492339+0000", 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015036699999999999 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492465+0000", 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125959 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.492675+0000", 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000209958 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.493287+0000", 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061244999999999995 2024-09-21T23:16:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499634+0000", 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0063463929999999997 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499725+0000", 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1125000000000006e-05 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.499788+0000", 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2644000000000005e-05 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:6416 3.2 3:42d25679:::rbd_data.10f1e443a6fa.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.640032689999998, 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054585550000000004, 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 6416 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619366+0000", 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619367+0000", 2024-09-21T23:16:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.207e-06 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619373+0000", 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.733e-06 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619388+0000", 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4843e-05 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.619397+0000", 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1050000000000001e-06 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620105+0000", 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000707795 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620198+0000", 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2881999999999996e-05 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.620290+0000", 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2503999999999994e-05 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.621413+0000", 2024-09-21T23:16:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011229650000000001 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624761+0000", 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033474529999999998 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624789+0000", 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.868e-05 2024-09-21T23:16:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:16.624825+0000", 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5388000000000001e-05 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:8815 3.0 3:0502be69:::rbd_data.10f1e443a6fa.00000000000001cb:head [set-alloc-hint object_size 524288 write_size 524288,write 188416~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.725268958999997, 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053156089999999998, 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8815 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534130+0000", 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534132+0000", 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.88e-06 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534152+0000", 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0224000000000002e-05 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534177+0000", 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5519000000000001e-05 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534192+0000", 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4684e-05 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534272+0000", 2024-09-21T23:16:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0091000000000003e-05 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534378+0000", 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105288 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.534516+0000", 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013853199999999999 2024-09-21T23:16:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.535863+0000", 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013471379999999999 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539301+0000", 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034381020000000002 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539371+0000", 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0130999999999997e-05 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.539445+0000", 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999999e-05 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:10277 3.2 3:40bd0fac:::rbd_data.10f1e443a6fa.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.270316192999999, 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052857490000000002, 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10277 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989083+0000", 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4799999999999997e-07 2024-09-21T23:16:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989089+0000", 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3979999999999998e-06 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989101+0000", 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2537999999999999e-05 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989118+0000", 2024-09-21T23:16:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6286999999999998e-05 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989181+0000", 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2853000000000003e-05 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989257+0000", 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6353000000000006e-05 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989329+0000", 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2497999999999996e-05 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.989757+0000", 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042811200000000002 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994316+0000", 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004558805 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994345+0000", 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8999000000000001e-05 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:17.994368+0000", 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3057999999999998e-05 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17427 3.4 3:223d3525:::rbd_data.10f1e443a6fa.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.42028981, 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048023049999999998, 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17427 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839109+0000", 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839110+0000", 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4199999999999996e-07 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839118+0000", 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839126+0000", 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5420000000000005e-06 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839132+0000", 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839207+0000", 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5210000000000007e-05 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839314+0000", 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010749 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839461+0000", 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000146601 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840367+0000", 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090656700000000005 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843802+0000", 2024-09-21T23:16:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034348410000000001 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843866+0000", 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4146999999999994e-05 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843911+0000", 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4811999999999997e-05 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17429 3.2 3:433fa519:::rbd_data.10f1e443a6fa.0000000000000077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.420248391999998, 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046812449999999997, 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17429 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839150+0000", 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839151+0000", 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839155+0000", 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8859999999999997e-06 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839165+0000", 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8870000000000004e-06 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.839185+0000", 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9887000000000001e-05 2024-09-21T23:16:11.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840907+0000", 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00172259 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.840969+0000", 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1851000000000001e-05 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.841059+0000", 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9604999999999993e-05 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843664+0000", 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026055570000000001 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843721+0000", 2024-09-21T23:16:11.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6382000000000003e-05 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843779+0000", 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8176e-05 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:20.843832+0000", 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2861000000000001e-05 2024-09-21T23:16:11.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4506.0:17711 3.0 3:0326984a:::rbd_data.10f1e443a6fa.000000000000018e:head [set-alloc-hint object_size 524288 write_size 524288,write 270336~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.238162465999999, 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049813679999999999, 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4506", 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/565803232", 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17711 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021236+0000", 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021239+0000", 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021268+0000", 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8883000000000001e-05 2024-09-21T23:16:11.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021299+0000", 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1106999999999999e-05 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021320+0000", 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1043000000000002e-05 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021493+0000", 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000172883 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.021776+0000", 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028353100000000001 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.022039+0000", 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026257899999999998 2024-09-21T23:16:11.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.023950+0000", 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019110850000000001 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.025958+0000", 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002007965 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026118+0000", 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016001899999999999 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.026218+0000", 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9625999999999998e-05 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.321 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4506.0:17959 3.1a e36/21 3:5f5ea830:::rbd_data.10f1e443a6fa.00000000000000b1:head v 36'103, mlcod=35'101)", 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.096220123000002, 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047440490000000002, 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163179+0000", 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163180+0000", 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7370000000000001e-06 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163206+0000", 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5103e-05 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163233+0000", 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7797000000000002e-05 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163253+0000", 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9621999999999999e-05 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163359+0000", 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010591599999999999 2024-09-21T23:16:11.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.163457+0000", 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7720000000000006e-05 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167864+0000", 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0044071639999999999 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:21.167923+0000", 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8990000000000003e-05 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:258 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.940353827999999, 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052566039999999998, 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.326 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 258 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319045+0000", 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319047+0000", 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8190000000000001e-06 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319055+0000", 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319074+0000", 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8822999999999999e-05 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319082+0000", 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.259e-06 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319155+0000", 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2893999999999997e-05 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319328+0000", 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017307499999999999 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.319555+0000", 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022681900000000001 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.320269+0000", 2024-09-21T23:16:11.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071382800000000001 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324226+0000", 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039570359999999997 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324266+0000", 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0491999999999998e-05 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.324302+0000", 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5206999999999997e-05 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:488 3.19 3:9f64ae25:::rbd_data.10f192b3a1cb.0000000000000134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.777623550999998, 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053403859999999999, 2024-09-21T23:16:11.332 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 488 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:11.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481775+0000", 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481778+0000", 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.802e-06 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481789+0000", 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0635000000000001e-05 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481820+0000", 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269000000000002e-05 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481836+0000", 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5620999999999998e-05 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.481984+0000", 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000148298 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482117+0000", 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013273500000000001 2024-09-21T23:16:11.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.482281+0000", 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016448600000000001 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.484057+0000", 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017759 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.486987+0000", 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029300300000000001 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487059+0000", 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:26.487116+0000", 2024-09-21T23:16:11.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6140999999999997e-05 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:907 3.2 3:473c643d:::rbd_data.10f192b3a1cb.0000000000000003:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.258198168, 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070017589999999998, 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 907 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001201+0000", 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001203+0000", 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5830000000000001e-06 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001216+0000", 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.235e-05 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001259+0000", 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3096999999999998e-05 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001282+0000", 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482999999999999e-05 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001464+0000", 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018153599999999999 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001871+0000", 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697999999999998 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002357+0000", 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048650800000000003 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006862+0000", 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045044129999999996 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.007957+0000", 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010950700000000001 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008095+0000", 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013839599999999999 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008202+0000", 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010734300000000001 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:909 3.b 3:d0dc65bb:::rbd_data.10f192b3a1cb.0000000000000004:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:11.346 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.258069691999999, 2024-09-21T23:16:11.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050996599999999998, 2024-09-21T23:16:11.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 909 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001329+0000", 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001330+0000", 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4399999999999998e-07 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001336+0000", 2024-09-21T23:16:11.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001359+0000", 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2807e-05 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001369+0000", 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0706e-05 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001491+0000", 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012197999999999999 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.001638+0000", 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014714599999999999 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.005983+0000", 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043442569999999998 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006137+0000", 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015446899999999999 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.006429+0000", 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029154199999999999 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:911 3.1a e39/21 3:5ee75455:::rbd_data.10f192b3a1cb.0000000000000005:head v 39'116, mlcod=38'113)", 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:11.352 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.257107988000001, 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060128619999999999, 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002291+0000", 2024-09-21T23:16:11.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002293+0000", 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9800000000000001e-06 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002316+0000", 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2753999999999999e-05 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002344+0000", 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8940000000000001e-05 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002364+0000", 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9038999999999998e-05 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002561+0000", 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019719900000000001 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.002611+0000", 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9975999999999997e-05 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008209+0000", 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055985310000000003 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.008304+0000", 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4443000000000004e-05 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4510.0:942 3.4 3:24ee8402:::rbd_data.10f192b3a1cb.0000000000000378:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.256096777000003, 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066099710000000001, 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4510", 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/4103853189", 2024-09-21T23:16:11.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 942 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003302+0000", 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003304+0000", 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003313+0000", 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003342+0000", 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8532999999999999e-05 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003362+0000", 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9942999999999999e-05 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003558+0000", 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019612899999999999 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.003771+0000", 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000212468 2024-09-21T23:16:11.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004242+0000", 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047164100000000002 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009111+0000", 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048689880000000003 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009644+0000", 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053264000000000002 2024-09-21T23:16:11.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009784+0000", 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014040600000000001 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009912+0000", 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000127756 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4510.0:916 3.a e39/25 3:552e5ef7:::rbd_data.10f192b3a1cb.0000000000000008:head v 39'79, mlcod=38'76)", 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.25501483, 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005313122, 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004384+0000", 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004386+0000", 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0269999999999998e-06 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004402+0000", 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6103000000000002e-05 2024-09-21T23:16:11.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004433+0000", 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0898999999999997e-05 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004451+0000", 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8167000000000001e-05 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004664+0000", 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000213146 2024-09-21T23:16:11.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.004716+0000", 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1400000000000003e-05 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009615+0000", 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048997040000000004 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:27.009697+0000", 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1675999999999995e-05 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4541.0:874 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.378436571999998, 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016068381999999999, 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4541", 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1644452617", 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 874 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880962+0000", 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880963+0000", 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7499999999999998e-07 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880969+0000", 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3299999999999998e-06 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880985+0000", 2024-09-21T23:16:11.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6393e-05 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.880993+0000", 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881058+0000", 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5501999999999997e-05 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881116+0000", 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7454000000000003e-05 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897031+0000", 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015914902000000002 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:4842 3.12 3:4e6a493b:::rbd_header.10f1163c318d:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-21T23:16:11.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.378097001999997, 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015900399999999999, 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4842 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881302+0000", 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881303+0000", 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5300000000000003e-07 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881307+0000", 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6110000000000004e-06 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881317+0000", 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8980000000000008e-06 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.881324+0000", 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5930000000000002e-06 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897054+0000", 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015730340999999998 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897105+0000", 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0454000000000002e-05 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:29.897202+0000", 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7650000000000002e-05 2024-09-21T23:16:11.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:596 3.12 3:4d026f6e:::rbd_data.10f1dc9013.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 638976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.308381990999997, 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078799430000000004, 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 596 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951017+0000", 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951020+0000", 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-06 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951929+0000", 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090880699999999997 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951965+0000", 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5734e-05 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.951982+0000", 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6806000000000001e-05 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952135+0000", 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015325899999999999 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.952560+0000", 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042476500000000002 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.953181+0000", 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062147299999999999 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.957005+0000", 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038238 2024-09-21T23:16:11.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958342+0000", 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001336622 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958557+0000", 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021509599999999999 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.958897+0000", 2024-09-21T23:16:11.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034014100000000002 2024-09-21T23:16:11.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.382 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:11.383 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-21T23:16:11.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:11.384 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:11.379+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:11.483 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:11.484+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:11.523 INFO:teuthology.orchestra.run.smithi170.stdout:2686 trunc from 0xdfad400 to 0xa2c2e00 2024-09-21T23:16:11.560 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:11.560 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:11.560 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:11.560 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:11.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:11.620 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:11.620+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:11.686 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:11.679+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:11.753 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:11.753 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:11.753 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:11.753 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:11.753 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:11.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:11.812 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:11.812+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:11.888 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:11.883+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.010334051000001, 2024-09-21T23:16:11.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:11.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.945 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.009313204000001, 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:11.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:11.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.009253530999999, 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.951 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:11.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.956 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.009163555000001, 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:11.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:11.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.962 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.009113644999999, 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:11.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:11.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:11.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.009057505000001, 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:11.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.009003722000003, 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:11.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:11.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.981 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008971445999997, 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:11.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:11.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.986 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008941166, 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:11.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008911509999997, 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:11.993 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:11.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008881543000001, 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008803714000003, 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008756007000002, 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008725810999998, 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008695893999999, 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008666093999999, 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008636160999998, 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008571197999999, 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008540721999999, 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.008511558000002, 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:12.056 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:12.057 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:12.058 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:12.056+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:12.158 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:12.156+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:12.189 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:12.189 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:12.189 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:12.189 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:12.203 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:12.233 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:16:12.300 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:12.300+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:12.321 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:12.322 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:12.322 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:12.322 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:12.322 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:12.337 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:12.359 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:12.360+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:12.460 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:12.460 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 86.238585162999996, 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.461 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:16:12.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:16:12.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.467 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.726179880000004, 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:16:12.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:16:12.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.726114976999995, 2024-09-21T23:16:12.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:12.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.725968538000004, 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.479 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:16:12.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:16:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:16:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.725899423000001, 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:16:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:16:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:16:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.725287097999995, 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:16:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:16:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.725082693000004, 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:16:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.725039906999996, 2024-09-21T23:16:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:16:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.334081619000003, 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:16:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:16:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333995109, 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:16:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:16:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333918220000001, 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333713380000006, 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:16:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333625097999999, 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:16:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:16:12.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:16:12.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333494270000003, 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.539 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:16:12.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:16:12.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:16:12.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333448617000002, 2024-09-21T23:16:12.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:16:12.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:16:12.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333410189999995, 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.550 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:16:12.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:16:12.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333280728000005, 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333259085999998, 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.564 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:12.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333239042000002, 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.570 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:16:12.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.333155050000002, 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:12.576 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:16:12.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:16:12.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:16:12.581 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:12.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:12.582 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:12.583 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:12.460+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:12.584 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:12.584 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:12.580+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:12.684 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:12.679+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:12.710 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:12.711 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:12.711 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:12.711 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:12.726 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:12.810 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:12.807+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:12.886 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:12.888+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:12.892 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:12.893 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:12.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:12.929 INFO:teuthology.orchestra.run.smithi170.stdout:2687 trunc from 0xa2c2e00 to 0xe75c200 2024-09-21T23:16:12.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:12.945 INFO:teuthology.orchestra.run.smithi170.stdout:2692 punch from 0xd2d2000 to 0xd2d8800, (0x6800 bytes) 2024-09-21T23:16:12.946 INFO:teuthology.orchestra.run.smithi170.stdout:2693 write 0x9829e00 thru 0x9837519 (0xd71a bytes) 2024-09-21T23:16:12.964 INFO:teuthology.orchestra.run.smithi170.stdout:2696 trunc from 0xe75c200 to 0x9ef2000 2024-09-21T23:16:12.994 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:12.991+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:13.045 INFO:teuthology.orchestra.run.smithi170.stdout:2700 write 0xa7fac00 thru 0xa806fee (0xc3ef bytes) 2024-09-21T23:16:13.047 INFO:teuthology.orchestra.run.smithi170.stdout:2702 read 0x4cc9600 thru 0x4ccd148 (0x3b49 bytes) 2024-09-21T23:16:13.048 INFO:teuthology.orchestra.run.smithi170.stdout:2704 write 0x798f400 thru 0x7996990 (0x7591 bytes) 2024-09-21T23:16:13.049 INFO:teuthology.orchestra.run.smithi170.stdout:2706 write 0x159ae00 thru 0x15a8071 (0xd272 bytes) 2024-09-21T23:16:13.052 INFO:teuthology.orchestra.run.smithi170.stdout:2707 read 0x3d4ee00 thru 0x3d50b5f (0x1d60 bytes) 2024-09-21T23:16:13.052 INFO:teuthology.orchestra.run.smithi170.stdout:2708 write 0x82e1200 thru 0x82eb790 (0xa591 bytes) 2024-09-21T23:16:13.054 INFO:teuthology.orchestra.run.smithi170.stdout:2709 write 0x9ad5a00 thru 0x9ae4172 (0xe773 bytes) 2024-09-21T23:16:13.085 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:13.085 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:13.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:13.085 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:13.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.085 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:13.085 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.127225008000003, 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:16:13.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:16:13.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.126958023999997, 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:16:13.092 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:16:13.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:16:13.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:16:13.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:16:13.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:16:13.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:16:13.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:16:13.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:13.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:16:13.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:16:13.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:16:13.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:16:13.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:13.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:16:13.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:16:13.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:16:13.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:16:13.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:16:13.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:16:13.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:16:13.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:16:13.102 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.102 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.103 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.103 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3357 3.11 3:8fb7664f:::rbd_data.10f1cfc9677.0000000000000015:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.103 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:13.103 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.908004180999999, 2024-09-21T23:16:13.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049070639999999999, 2024-09-21T23:16:13.104 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.104 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.104 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.104 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.104 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.104 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.105 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3357 2024-09-21T23:16:13.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.105 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:13.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173027+0000", 2024-09-21T23:16:13.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173029+0000", 2024-09-21T23:16:13.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.494e-06 2024-09-21T23:16:13.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173037+0000", 2024-09-21T23:16:13.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1929999999999993e-06 2024-09-21T23:16:13.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173060+0000", 2024-09-21T23:16:13.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2717999999999998e-05 2024-09-21T23:16:13.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.173074+0000", 2024-09-21T23:16:13.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4671000000000001e-05 2024-09-21T23:16:13.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204211+0000", 2024-09-21T23:16:13.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031136891999999999 2024-09-21T23:16:13.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.204347+0000", 2024-09-21T23:16:13.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013595400000000001 2024-09-21T23:16:13.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222098+0000", 2024-09-21T23:16:13.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017750717999999999 2024-09-21T23:16:13.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.112 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.113 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3418 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.113 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:13.113 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899985417, 2024-09-21T23:16:13.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054653000000003, 2024-09-21T23:16:13.114 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.114 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.114 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3418 2024-09-21T23:16:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:13.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181046+0000", 2024-09-21T23:16:13.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181047+0000", 2024-09-21T23:16:13.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6899999999999996e-07 2024-09-21T23:16:13.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181057+0000", 2024-09-21T23:16:13.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0223e-05 2024-09-21T23:16:13.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181064+0000", 2024-09-21T23:16:13.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9949999999999999e-06 2024-09-21T23:16:13.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181069+0000", 2024-09-21T23:16:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4160000000000003e-06 2024-09-21T23:16:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.221944+0000", 2024-09-21T23:16:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040874684000000001 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222101+0000", 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015656599999999999 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3419 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899954423000001, 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041067436999999998, 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3419 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181077+0000", 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181080+0000", 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.402e-06 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181087+0000", 2024-09-21T23:16:13.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181092+0000", 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5720000000000002e-06 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222136+0000", 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041044316999999997 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222144+0000", 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9349999999999994e-06 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3420 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899932, 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041092287999999998, 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.126 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3420 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181099+0000", 2024-09-21T23:16:13.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181100+0000", 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8500000000000001e-07 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181102+0000", 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2879999999999999e-06 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181109+0000", 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7830000000000001e-06 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181115+0000", 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7819999999999999e-06 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222181+0000", 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041065989999999997 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222192+0000", 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.076e-05 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3421 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899909502, 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041102946000000001, 2024-09-21T23:16:13.130 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3421 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181122+0000", 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181125+0000", 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5390000000000002e-06 2024-09-21T23:16:13.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181132+0000", 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6610000000000003e-06 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181137+0000", 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2429999999999996e-06 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222217+0000", 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041080113000000001 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222225+0000", 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9049999999999997e-06 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3422 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899886921, 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041150084000000003, 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.135 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3422 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181144+0000", 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181145+0000", 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181148+0000", 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.57e-06 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181154+0000", 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9909999999999999e-06 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181160+0000", 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222287+0000", 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041126708999999997 2024-09-21T23:16:13.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222295+0000", 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6799999999999993e-06 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3424 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899864050000001, 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041160906999999997, 2024-09-21T23:16:13.139 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3424 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181167+0000", 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181168+0000", 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0600000000000001e-07 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181170+0000", 2024-09-21T23:16:13.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6570000000000001e-06 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181178+0000", 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0829999999999998e-06 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181183+0000", 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3410000000000002e-06 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222321+0000", 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041137765 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222328+0000", 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6550000000000004e-06 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3425 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.143 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899841195, 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041445086999999999, 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3425 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181190+0000", 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181191+0000", 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999999e-07 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181193+0000", 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6800000000000002e-06 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181200+0000", 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6760000000000001e-06 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181205+0000", 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.383e-06 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222627+0000", 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041421342 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222635+0000", 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6130000000000004e-06 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.147 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3509 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899460551000001, 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040950634, 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3509 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181571+0000", 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181574+0000", 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.199e-06 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181580+0000", 2024-09-21T23:16:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0560000000000001e-06 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181585+0000", 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1669999999999998e-06 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222516+0000", 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040931568000000002 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222521+0000", 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1499999999999998e-06 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3510 3.19 3.7fecfd19 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899437809999998, 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040943715999999998, 2024-09-21T23:16:13.152 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3510 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181594+0000", 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.32e-07 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181596+0000", 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.339e-06 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181603+0000", 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5599999999999999e-06 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181608+0000", 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-21T23:16:13.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222534+0000", 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040925221999999997 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222537+0000", 2024-09-21T23:16:13.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.681e-06 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4666.0:3567 3.1 3.d9b65201 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.899074676000001, 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040956498000000001, 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.157 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4666", 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3517949127", 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 3567 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181957+0000", 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7899999999999999e-07 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181959+0000", 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1830000000000001e-06 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181966+0000", 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3119999999999999e-06 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.181973+0000", 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1099999999999997e-06 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222902+0000", 2024-09-21T23:16:13.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040928961999999999 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:41.222913+0000", 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1552000000000001e-05 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.161 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:499 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.301120029, 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041124135999999999, 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 499 2024-09-21T23:16:13.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779911+0000", 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779913+0000", 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1629999999999999e-06 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779918+0000", 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779938+0000", 2024-09-21T23:16:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9553000000000001e-05 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779957+0000", 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8876999999999999e-05 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820711+0000", 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040754603 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.820848+0000", 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013633699999999999 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821036+0000", 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018796899999999999 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.166 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:500 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.301054889, 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041461570000000003, 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 500 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779977+0000", 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779978+0000", 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1969999999999999e-06 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.779983+0000", 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780045+0000", 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2082000000000006e-05 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780087+0000", 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1863000000000002e-05 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821117+0000", 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041030076999999998 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821247+0000", 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012996199999999999 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821438+0000", 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019079899999999999 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.171 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:501 3.b 3:d5cd4cb9:::rbd_data.10f11f68eed1.0000000000000009:e [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.300861157, 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042023224999999997, 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.172 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 501 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780170+0000", 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780172+0000", 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2069999999999998e-06 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780184+0000", 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1039999999999999e-05 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780215+0000", 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1006000000000001e-05 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780235+0000", 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0483999999999999e-05 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821553+0000", 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041318311000000003 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821848+0000", 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029481199999999998 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822193+0000", 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034536499999999998 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.176 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:506 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.300762603999999, 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041920526, 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:13.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 506 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780269+0000", 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780270+0000", 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3370000000000001e-06 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780278+0000", 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8730000000000007e-06 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780300+0000", 2024-09-21T23:16:13.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2183000000000001e-05 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780316+0000", 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5413e-05 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821521+0000", 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041205506000000003 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.821802+0000", 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028134500000000001 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822189+0000", 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00038686900000000003 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:507 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:13.181 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.300692737999999, 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042296304, 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 507 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780339+0000", 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780340+0000", 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.31e-06 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780347+0000", 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780367+0000", 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9466e-05 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780382+0000", 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5274e-05 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822269+0000", 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041886592 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822418+0000", 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014924299999999999 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822635+0000", 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021715499999999999 2024-09-21T23:16:13.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.186 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.186 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:508 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:13.186 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.300552284999998, 2024-09-21T23:16:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042742988000000003, 2024-09-21T23:16:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 508 2024-09-21T23:16:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:13.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780479+0000", 2024-09-21T23:16:13.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780481+0000", 2024-09-21T23:16:13.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4869999999999999e-06 2024-09-21T23:16:13.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.780489+0000", 2024-09-21T23:16:13.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1200000000000002e-06 2024-09-21T23:16:13.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781280+0000", 2024-09-21T23:16:13.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000791583 2024-09-21T23:16:13.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781307+0000", 2024-09-21T23:16:13.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6486999999999999e-05 2024-09-21T23:16:13.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822728+0000", 2024-09-21T23:16:13.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041420917000000002 2024-09-21T23:16:13.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.822937+0000", 2024-09-21T23:16:13.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000209593 2024-09-21T23:16:13.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823222+0000", 2024-09-21T23:16:13.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000284801 2024-09-21T23:16:13.195 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.195 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.196 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4810.0:509 3.1b 3:d8e1f4d0:::rbd_data.10f11f68eed1.000000000000000e:e [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T23:16:13.196 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:13.196 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.299306819999998, 2024-09-21T23:16:13.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041990254999999997, 2024-09-21T23:16:13.197 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:13.197 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:13.197 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:13.197 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:13.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4810", 2024-09-21T23:16:13.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3623350896", 2024-09-21T23:16:13.198 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 509 2024-09-21T23:16:13.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.198 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:13.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:13.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:13.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:13.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781725+0000", 2024-09-21T23:16:13.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:13.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:13.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781726+0000", 2024-09-21T23:16:13.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2759999999999999e-06 2024-09-21T23:16:13.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:13.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781744+0000", 2024-09-21T23:16:13.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8377e-05 2024-09-21T23:16:13.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:13.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781767+0000", 2024-09-21T23:16:13.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3006999999999998e-05 2024-09-21T23:16:13.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:13.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.781780+0000", 2024-09-21T23:16:13.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2393e-05 2024-09-21T23:16:13.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:13.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823275+0000", 2024-09-21T23:16:13.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041494879999999998 2024-09-21T23:16:13.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:13.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823485+0000", 2024-09-21T23:16:13.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021036300000000001 2024-09-21T23:16:13.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:13.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:13.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:13.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:52.823715+0000", 2024-09-21T23:16:13.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022995900000000001 2024-09-21T23:16:13.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.206 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:13.206 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:13.206 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:13.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:16:13.209 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:13.209 INFO:teuthology.orchestra.run.smithi170.stdout:2710 trunc from 0xa807000 to 0x3caae00 2024-09-21T23:16:13.210 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:13.210 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:13.211 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:13.211 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:13.212 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:13.203+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:13.212 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:13.213 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:13.225 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:13.225 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:13.252 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:13.252 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:13.272 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:13.272 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:13.332 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:13.332+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:13.387 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:13.388 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:13.403 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:13.409 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:13.408+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:13.510 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:13.508+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:13.529 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:13.529 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:13.544 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:13.611 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:13.607+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:13.615 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:16:13.640 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-21T23:16:12.866533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":241645534,"num_objects":608,"num_object_clones":137,"num_object_copies":1216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":260,"num_read":101742,"num_read_kb":340631,"num_write":10047,"num_write_kb":52986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":778,"num_bytes_recovered":7208077,"num_keys_recovered":95,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8769,"ondisk_log_size":8769,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":197600,"kb_used_data":45332,"kb_used_omap":10,"kb_used_meta":152181,"kb_avail":377289760,"statfs":{"total":386547056640,"available":386344714240,"internally_reserved":0,"allocated":46419968,"data_stored":53384092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":155833972},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7265170,"num_objects":157,"num_object_clones":22,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":117,"num_read":11814,"num_read_kb":19498,"num_write":268,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":5458207,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.659193"},"pg_stats":[{"pgid":"3.1f","version":"68'194","reported_seq":439,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:11.180073+0000","last_change":"2024-09-21T23:16:06.209669+0000","last_active":"2024-09-21T23:16:11.180073+0000","last_peered":"2024-09-21T23:16:11.180073+0000","last_clean":"2024-09-21T23:16:11.180073+0000","last_became_active":"2024-09-21T23:16:06.208917+0000","last_became_peered":"2024-09-21T23:16:06.208917+0000","last_unstale":"2024-09-21T23:16:11.180073+0000","last_undegraded":"2024-09-21T23:16:11.180073+0000","last_fullsized":"2024-09-21T23:16:11.180073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:11.232320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":6582879,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":452,"num_read_kb":513,"num_write":232,"num_write_kb":646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":14388,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'187","reported_seq":1020,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.491226+0000","last_change":"2024-09-21T23:15:35.968961+0000","last_active":"2024-09-21T23:16:09.491226+0000","last_peered":"2024-09-21T23:16:09.491226+0000","last_clean":"2024-09-21T23:16:09.491226+0000","last_became_active":"2024-09-21T23:15:35.968136+0000","last_became_peered":"2024-09-21T23:15:35.968136+0000","last_unstale":"2024-09-21T23:16:09.491226+0000","last_undegraded":"2024-09-21T23:16:09.491226+0000","last_fullsized":"2024-09-21T23:16:09.491226+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:58.125595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":8741428,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":711,"num_read_kb":9274,"num_write":177,"num_write_kb":3029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"68'91","reported_seq":1225,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.654650+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:16:09.654650+0000","last_peered":"2024-09-21T23:16:09.654650+0000","last_clean":"2024-09-21T23:16:09.654650+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:16:09.654650+0000","last_undegraded":"2024-09-21T23:16:09.654650+0000","last_fullsized":"2024-09-21T23:16:09.654650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:47.348531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":4448768,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":1036,"num_read_kb":5876,"num_write":87,"num_write_kb":1986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"68'232","reported_seq":2553,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.821368+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:16:09.821368+0000","last_peered":"2024-09-21T23:16:09.821368+0000","last_clean":"2024-09-21T23:16:09.821368+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:16:09.821368+0000","last_undegraded":"2024-09-21T23:16:09.821368+0000","last_fullsized":"2024-09-21T23:16:09.821368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:30:21.228932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":10925632,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":2669,"num_read_kb":16929,"num_write":213,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"68'244","reported_seq":2050,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:10.602372+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:16:10.602372+0000","last_peered":"2024-09-21T23:16:10.602372+0000","last_clean":"2024-09-21T23:16:10.602372+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:16:10.602372+0000","last_undegraded":"2024-09-21T23:16:10.602372+0000","last_fullsized":"2024-09-21T23:16:10.602372+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:28:37.945699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":5896734,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":2692,"num_read_kb":11595,"num_write":259,"num_write_kb":876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"68'289","reported_seq":5241,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.823483+0000","last_change":"2024-09-21T23:15:35.969265+0000","last_active":"2024-09-21T23:16:09.823483+0000","last_peered":"2024-09-21T23:16:09.823483+0000","last_clean":"2024-09-21T23:16:09.823483+0000","last_became_active":"2024-09-21T23:15:35.967793+0000","last_became_peered":"2024-09-21T23:15:35.967793+0000","last_unstale":"2024-09-21T23:16:09.823483+0000","last_undegraded":"2024-09-21T23:16:09.823483+0000","last_fullsized":"2024-09-21T23:16:09.823483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:57:03.416081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":4733475,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":6629,"num_read_kb":16528,"num_write":211,"num_write_kb":762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":39443,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"68'128","reported_seq":1157,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.820709+0000","last_change":"2024-09-21T23:15:32.983007+0000","last_active":"2024-09-21T23:16:09.820709+0000","last_peered":"2024-09-21T23:16:09.820709+0000","last_clean":"2024-09-21T23:16:09.820709+0000","last_became_active":"2024-09-21T23:15:32.948191+0000","last_became_peered":"2024-09-21T23:15:32.948191+0000","last_unstale":"2024-09-21T23:16:09.820709+0000","last_undegraded":"2024-09-21T23:16:09.820709+0000","last_fullsized":"2024-09-21T23:16:09.820709+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:17:52.523144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":5218832,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":896,"num_read_kb":6343,"num_write":121,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"68'330","reported_seq":2105,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.647147+0000","last_change":"2024-09-21T23:16:06.209284+0000","last_active":"2024-09-21T23:16:09.647147+0000","last_peered":"2024-09-21T23:16:09.647147+0000","last_clean":"2024-09-21T23:16:09.647147+0000","last_became_active":"2024-09-21T23:16:06.208604+0000","last_became_peered":"2024-09-21T23:16:06.208604+0000","last_unstale":"2024-09-21T23:16:09.647147+0000","last_undegraded":"2024-09-21T23:16:09.647147+0000","last_fullsized":"2024-09-21T23:16:09.647147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:55:14.398341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":11246652,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2708,"num_read_kb":13644,"num_write":486,"num_write_kb":2851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":75324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"68'170","reported_seq":4709,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:10.608169+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:16:10.608169+0000","last_peered":"2024-09-21T23:16:10.608169+0000","last_clean":"2024-09-21T23:16:10.608169+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:16:10.608169+0000","last_undegraded":"2024-09-21T23:16:10.608169+0000","last_fullsized":"2024-09-21T23:16:10.608169+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:02.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":11476032,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":4421,"num_read_kb":8792,"num_write":157,"num_write_kb":683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"68'296","reported_seq":1602,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.811267+0000","last_change":"2024-09-21T23:15:35.965248+0000","last_active":"2024-09-21T23:16:09.811267+0000","last_peered":"2024-09-21T23:16:09.811267+0000","last_clean":"2024-09-21T23:16:09.811267+0000","last_became_active":"2024-09-21T23:15:35.963512+0000","last_became_peered":"2024-09-21T23:15:35.963512+0000","last_unstale":"2024-09-21T23:16:09.811267+0000","last_undegraded":"2024-09-21T23:16:09.811267+0000","last_fullsized":"2024-09-21T23:16:09.811267+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:13:51.314588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":17430178,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":1918,"num_read_kb":6820,"num_write":390,"num_write_kb":2126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":94370,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"68'117","reported_seq":2926,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.520949+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:16:09.520949+0000","last_peered":"2024-09-21T23:16:09.520949+0000","last_clean":"2024-09-21T23:16:09.520949+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:16:09.520949+0000","last_undegraded":"2024-09-21T23:16:09.520949+0000","last_fullsized":"2024-09-21T23:16:09.520949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:32:05.282874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898304,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":2711,"num_read_kb":24295,"num_write":115,"num_write_kb":2601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"68'486","reported_seq":1811,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.238307+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:16:09.238307+0000","last_peered":"2024-09-21T23:16:09.238307+0000","last_clean":"2024-09-21T23:16:09.238307+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:16:09.238307+0000","last_undegraded":"2024-09-21T23:16:09.238307+0000","last_fullsized":"2024-09-21T23:16:09.238307+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:22:12.498326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8018079,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":3011,"num_read_kb":11678,"num_write":729,"num_write_kb":1403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.237873+0000","last_change":"2024-09-21T23:15:06.350164+0000","last_active":"2024-09-21T23:16:09.237873+0000","last_peered":"2024-09-21T23:16:09.237873+0000","last_clean":"2024-09-21T23:16:09.237873+0000","last_became_active":"2024-09-21T23:15:06.349271+0000","last_became_peered":"2024-09-21T23:15:06.349271+0000","last_unstale":"2024-09-21T23:16:09.237873+0000","last_undegraded":"2024-09-21T23:16:09.237873+0000","last_fullsized":"2024-09-21T23:16:09.237873+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:56.638782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"68'169","reported_seq":4859,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:10.632976+0000","last_change":"2024-09-21T23:16:06.216189+0000","last_active":"2024-09-21T23:16:10.632976+0000","last_peered":"2024-09-21T23:16:10.632976+0000","last_clean":"2024-09-21T23:16:10.632976+0000","last_became_active":"2024-09-21T23:16:06.215526+0000","last_became_peered":"2024-09-21T23:16:06.215526+0000","last_unstale":"2024-09-21T23:16:10.632976+0000","last_undegraded":"2024-09-21T23:16:10.632976+0000","last_fullsized":"2024-09-21T23:16:10.632976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:42:54.577597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":20174888,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":4538,"num_read_kb":9499,"num_write":160,"num_write_kb":2077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1339432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.238483+0000","last_change":"2024-09-21T23:16:04.189267+0000","last_active":"2024-09-21T23:16:09.238483+0000","last_peered":"2024-09-21T23:16:09.238483+0000","last_clean":"2024-09-21T23:16:09.238483+0000","last_became_active":"2024-09-21T23:16:04.189021+0000","last_became_peered":"2024-09-21T23:16:04.189021+0000","last_unstale":"2024-09-21T23:16:09.238483+0000","last_undegraded":"2024-09-21T23:16:09.238483+0000","last_fullsized":"2024-09-21T23:16:09.238483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:19.263980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"68'1470","reported_seq":4166,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.815145+0000","last_change":"2024-09-21T23:15:35.969593+0000","last_active":"2024-09-21T23:16:09.815145+0000","last_peered":"2024-09-21T23:16:09.815145+0000","last_clean":"2024-09-21T23:16:09.815145+0000","last_became_active":"2024-09-21T23:15:35.967156+0000","last_became_peered":"2024-09-21T23:15:35.967156+0000","last_unstale":"2024-09-21T23:16:09.815145+0000","last_undegraded":"2024-09-21T23:16:09.815145+0000","last_fullsized":"2024-09-21T23:16:09.815145+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:57:08.860092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":7646122,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":9743,"num_read_kb":28640,"num_write":2646,"num_write_kb":5251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":247168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.311775+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:16:09.311775+0000","last_peered":"2024-09-21T23:16:09.311775+0000","last_clean":"2024-09-21T23:16:09.311775+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:16:09.311775+0000","last_undegraded":"2024-09-21T23:16:09.311775+0000","last_fullsized":"2024-09-21T23:16:09.311775+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"68'124","reported_seq":778,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.710791+0000","last_change":"2024-09-21T23:16:06.210143+0000","last_active":"2024-09-21T23:16:09.710791+0000","last_peered":"2024-09-21T23:16:09.710791+0000","last_clean":"2024-09-21T23:16:09.710791+0000","last_became_active":"2024-09-21T23:16:06.209634+0000","last_became_peered":"2024-09-21T23:16:06.209634+0000","last_unstale":"2024-09-21T23:16:09.710791+0000","last_undegraded":"2024-09-21T23:16:09.710791+0000","last_fullsized":"2024-09-21T23:16:09.710791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:35.323035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":4251136,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":989,"num_read_kb":1691,"num_write":103,"num_write_kb":439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":52736,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.311622+0000","last_change":"2024-09-21T23:16:04.198942+0000","last_active":"2024-09-21T23:16:09.311622+0000","last_peered":"2024-09-21T23:16:09.311622+0000","last_clean":"2024-09-21T23:16:09.311622+0000","last_became_active":"2024-09-21T23:16:04.198333+0000","last_became_peered":"2024-09-21T23:16:04.198333+0000","last_unstale":"2024-09-21T23:16:09.311622+0000","last_undegraded":"2024-09-21T23:16:09.311622+0000","last_fullsized":"2024-09-21T23:16:09.311622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:55:59.745128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"68'288","reported_seq":2329,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.857978+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:16:09.857978+0000","last_peered":"2024-09-21T23:16:09.857978+0000","last_clean":"2024-09-21T23:16:09.857978+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:16:09.857978+0000","last_undegraded":"2024-09-21T23:16:09.857978+0000","last_fullsized":"2024-09-21T23:16:09.857978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:20:34.841501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":9861959,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":3284,"num_read_kb":14386,"num_write":332,"num_write_kb":1518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'190","reported_seq":1899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.857323+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:16:09.857323+0000","last_peered":"2024-09-21T23:16:09.857323+0000","last_clean":"2024-09-21T23:16:09.857323+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:16:09.857323+0000","last_undegraded":"2024-09-21T23:16:09.857323+0000","last_fullsized":"2024-09-21T23:16:09.857323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:20:54.464445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":7304734,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":1643,"num_read_kb":10737,"num_write":197,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":130,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.240633+0000","last_change":"2024-09-21T23:16:04.205303+0000","last_active":"2024-09-21T23:16:09.240633+0000","last_peered":"2024-09-21T23:16:09.240633+0000","last_clean":"2024-09-21T23:16:09.240633+0000","last_became_active":"2024-09-21T23:16:04.181322+0000","last_became_peered":"2024-09-21T23:16:04.181322+0000","last_unstale":"2024-09-21T23:16:09.240633+0000","last_undegraded":"2024-09-21T23:16:09.240633+0000","last_fullsized":"2024-09-21T23:16:09.240633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:46:04.294981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"68'184","reported_seq":2379,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.867059+0000","last_change":"2024-09-21T23:15:35.969320+0000","last_active":"2024-09-21T23:16:09.867059+0000","last_peered":"2024-09-21T23:16:09.867059+0000","last_clean":"2024-09-21T23:16:09.867059+0000","last_became_active":"2024-09-21T23:15:35.968596+0000","last_became_peered":"2024-09-21T23:15:35.968596+0000","last_unstale":"2024-09-21T23:16:09.867059+0000","last_undegraded":"2024-09-21T23:16:09.867059+0000","last_fullsized":"2024-09-21T23:16:09.867059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:09:43.157456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":3327604,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":1925,"num_read_kb":7075,"num_write":169,"num_write_kb":690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":29200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.237687+0000","last_change":"2024-09-21T23:16:04.197581+0000","last_active":"2024-09-21T23:16:09.237687+0000","last_peered":"2024-09-21T23:16:09.237687+0000","last_clean":"2024-09-21T23:16:09.237687+0000","last_became_active":"2024-09-21T23:16:04.196985+0000","last_became_peered":"2024-09-21T23:16:04.196985+0000","last_unstale":"2024-09-21T23:16:09.237687+0000","last_undegraded":"2024-09-21T23:16:09.237687+0000","last_fullsized":"2024-09-21T23:16:09.237687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:24:15.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"68'214","reported_seq":1396,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:10.569827+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:16:10.569827+0000","last_peered":"2024-09-21T23:16:10.569827+0000","last_clean":"2024-09-21T23:16:10.569827+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:16:10.569827+0000","last_undegraded":"2024-09-21T23:16:10.569827+0000","last_fullsized":"2024-09-21T23:16:10.569827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:09.857196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":4481060,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1135,"num_read_kb":6712,"num_write":209,"num_write_kb":3814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.311188+0000","last_change":"2024-09-21T23:16:04.207675+0000","last_active":"2024-09-21T23:16:09.311188+0000","last_peered":"2024-09-21T23:16:09.311188+0000","last_clean":"2024-09-21T23:16:09.311188+0000","last_became_active":"2024-09-21T23:16:04.206739+0000","last_became_peered":"2024-09-21T23:16:04.206739+0000","last_unstale":"2024-09-21T23:16:09.311188+0000","last_undegraded":"2024-09-21T23:16:09.311188+0000","last_fullsized":"2024-09-21T23:16:09.311188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:12.680088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"68'194","reported_seq":646,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.821565+0000","last_change":"2024-09-21T23:15:33.016643+0000","last_active":"2024-09-21T23:16:09.821565+0000","last_peered":"2024-09-21T23:16:09.821565+0000","last_clean":"2024-09-21T23:16:09.821565+0000","last_became_active":"2024-09-21T23:15:32.955127+0000","last_became_peered":"2024-09-21T23:15:32.955127+0000","last_unstale":"2024-09-21T23:16:09.821565+0000","last_undegraded":"2024-09-21T23:16:09.821565+0000","last_fullsized":"2024-09-21T23:16:09.821565+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:12:35.747830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":13215344,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":468,"num_read_kb":3898,"num_write":200,"num_write_kb":1839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":10301,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.241067+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:16:09.241067+0000","last_peered":"2024-09-21T23:16:09.241067+0000","last_clean":"2024-09-21T23:16:09.241067+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:16:09.241067+0000","last_undegraded":"2024-09-21T23:16:09.241067+0000","last_fullsized":"2024-09-21T23:16:09.241067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.311279+0000","last_change":"2024-09-21T23:16:04.207237+0000","last_active":"2024-09-21T23:16:09.311279+0000","last_peered":"2024-09-21T23:16:09.311279+0000","last_clean":"2024-09-21T23:16:09.311279+0000","last_became_active":"2024-09-21T23:16:04.206117+0000","last_became_peered":"2024-09-21T23:16:04.206117+0000","last_unstale":"2024-09-21T23:16:09.311279+0000","last_undegraded":"2024-09-21T23:16:09.311279+0000","last_fullsized":"2024-09-21T23:16:09.311279+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:26:34.280570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"68'515","reported_seq":9832,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.815608+0000","last_change":"2024-09-21T23:15:35.968122+0000","last_active":"2024-09-21T23:16:09.815608+0000","last_peered":"2024-09-21T23:16:09.815608+0000","last_clean":"2024-09-21T23:16:09.815608+0000","last_became_active":"2024-09-21T23:15:35.967154+0000","last_became_peered":"2024-09-21T23:15:35.967154+0000","last_unstale":"2024-09-21T23:16:09.815608+0000","last_undegraded":"2024-09-21T23:16:09.815608+0000","last_fullsized":"2024-09-21T23:16:09.815608+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:25:43.806979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":3729936,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":10529,"num_read_kb":14824,"num_write":304,"num_write_kb":1362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":15888,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"68'301","reported_seq":2880,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.857995+0000","last_change":"2024-09-21T23:15:03.325366+0000","last_active":"2024-09-21T23:16:09.857995+0000","last_peered":"2024-09-21T23:16:09.857995+0000","last_clean":"2024-09-21T23:16:09.857995+0000","last_became_active":"2024-09-21T23:15:01.329082+0000","last_became_peered":"2024-09-21T23:15:01.329082+0000","last_unstale":"2024-09-21T23:16:09.857995+0000","last_undegraded":"2024-09-21T23:16:09.857995+0000","last_fullsized":"2024-09-21T23:16:09.857995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:23:11.928598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":5206528,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":4296,"num_read_kb":7937,"num_write":217,"num_write_kb":865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"68'138","reported_seq":468,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.808227+0000","last_change":"2024-09-21T23:15:33.025560+0000","last_active":"2024-09-21T23:16:09.808227+0000","last_peered":"2024-09-21T23:16:09.808227+0000","last_clean":"2024-09-21T23:16:09.808227+0000","last_became_active":"2024-09-21T23:15:32.949370+0000","last_became_peered":"2024-09-21T23:15:32.949370+0000","last_unstale":"2024-09-21T23:16:09.808227+0000","last_undegraded":"2024-09-21T23:16:09.808227+0000","last_fullsized":"2024-09-21T23:16:09.808227+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:44:05.640913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2480656,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":207,"num_read_kb":1014,"num_write":131,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":8192,"num_keys_recovered":3,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"68'323","reported_seq":3813,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:11.115155+0000","last_change":"2024-09-21T23:16:07.221006+0000","last_active":"2024-09-21T23:16:11.115155+0000","last_peered":"2024-09-21T23:16:11.115155+0000","last_clean":"2024-09-21T23:16:11.115155+0000","last_became_active":"2024-09-21T23:16:07.219661+0000","last_became_peered":"2024-09-21T23:16:07.219661+0000","last_unstale":"2024-09-21T23:16:11.115155+0000","last_undegraded":"2024-09-21T23:16:11.115155+0000","last_fullsized":"2024-09-21T23:16:11.115155+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:08:45.522566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":4011551,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":5493,"num_read_kb":14238,"num_write":225,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":314911,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"68'217","reported_seq":707,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.485035+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:16:09.485035+0000","last_peered":"2024-09-21T23:16:09.485035+0000","last_clean":"2024-09-21T23:16:09.485035+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:16:09.485035+0000","last_undegraded":"2024-09-21T23:16:09.485035+0000","last_fullsized":"2024-09-21T23:16:09.485035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:03.168714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":5405229,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":376,"num_read_kb":1023,"num_write":219,"num_write_kb":1090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"68'188","reported_seq":3270,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.825194+0000","last_change":"2024-09-21T23:15:06.357124+0000","last_active":"2024-09-21T23:16:09.825194+0000","last_peered":"2024-09-21T23:16:09.825194+0000","last_clean":"2024-09-21T23:16:09.825194+0000","last_became_active":"2024-09-21T23:15:06.356745+0000","last_became_peered":"2024-09-21T23:15:06.356745+0000","last_unstale":"2024-09-21T23:16:09.825194+0000","last_undegraded":"2024-09-21T23:16:09.825194+0000","last_fullsized":"2024-09-21T23:16:09.825194+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:13:25.234179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5796620,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":3258,"num_read_kb":23550,"num_write":220,"num_write_kb":1122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"68'197","reported_seq":1937,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.808982+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:16:09.808982+0000","last_peered":"2024-09-21T23:16:09.808982+0000","last_clean":"2024-09-21T23:16:09.808982+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:16:09.808982+0000","last_undegraded":"2024-09-21T23:16:09.808982+0000","last_fullsized":"2024-09-21T23:16:09.808982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:50:45.478223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":4714642,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":2730,"num_read_kb":7709,"num_write":197,"num_write_kb":765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"68'147","reported_seq":2052,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.485987+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:16:09.485987+0000","last_peered":"2024-09-21T23:16:09.485987+0000","last_clean":"2024-09-21T23:16:09.485987+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:16:09.485987+0000","last_undegraded":"2024-09-21T23:16:09.485987+0000","last_fullsized":"2024-09-21T23:16:09.485987+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:02:36.562891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":5008400,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":2057,"num_read_kb":6536,"num_write":121,"num_write_kb":494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"68'180","reported_seq":1670,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.818656+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:16:09.818656+0000","last_peered":"2024-09-21T23:16:09.818656+0000","last_clean":"2024-09-21T23:16:09.818656+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:16:09.818656+0000","last_undegraded":"2024-09-21T23:16:09.818656+0000","last_fullsized":"2024-09-21T23:16:09.818656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:51:54.903408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":6259200,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":1294,"num_read_kb":9308,"num_write":171,"num_write_kb":1026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"68'332","reported_seq":1821,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.477259+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:16:09.477259+0000","last_peered":"2024-09-21T23:16:09.477259+0000","last_clean":"2024-09-21T23:16:09.477259+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:16:09.477259+0000","last_undegraded":"2024-09-21T23:16:09.477259+0000","last_fullsized":"2024-09-21T23:16:09.477259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:16.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2856485,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":2457,"num_read_kb":5407,"num_write":492,"num_write_kb":933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"68'349","reported_seq":6166,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:11.182242+0000","last_change":"2024-09-21T23:16:06.207983+0000","last_active":"2024-09-21T23:16:11.182242+0000","last_peered":"2024-09-21T23:16:11.182242+0000","last_clean":"2024-09-21T23:16:11.182242+0000","last_became_active":"2024-09-21T23:16:06.206978+0000","last_became_peered":"2024-09-21T23:16:06.206978+0000","last_unstale":"2024-09-21T23:16:11.182242+0000","last_undegraded":"2024-09-21T23:16:11.182242+0000","last_fullsized":"2024-09-21T23:16:11.182242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:08.720141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":10283580,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":7914,"num_read_kb":17645,"num_write":243,"num_write_kb":1156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":154172,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"68'165","reported_seq":7682,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T23:16:09.707612+0000","last_change":"2024-09-21T23:15:06.345498+0000","last_active":"2024-09-21T23:16:09.707612+0000","last_peered":"2024-09-21T23:16:09.707612+0000","last_clean":"2024-09-21T23:16:09.707612+0000","last_became_active":"2024-09-21T23:15:06.344491+0000","last_became_peered":"2024-09-21T23:15:06.344491+0000","last_unstale":"2024-09-21T23:16:09.707612+0000","last_undegraded":"2024-09-21T23:16:09.707612+0000","last_fullsized":"2024-09-21T23:16:09.707612+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:13:33.610595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7422480,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":7436,"num_read_kb":16415,"num_write":153,"num_write_kb":763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":241055147,"num_objects":604,"num_object_clones":137,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":604,"num_whiteouts":260,"num_read":101626,"num_read_kb":340531,"num_write":9886,"num_write_kb":50568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":775,"num_bytes_recovered":6617690,"num_keys_recovered":95,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43921408,"data_stored":52677609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":8649,"ondisk_log_size":8649,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52396,"kb_used_data":12888,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319444,"statfs":{"total":96636764160,"available":96583110656,"internally_reserved":0,"allocated":13197312,"data_stored":16343827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43052,"kb_used_data":16024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328788,"statfs":{"total":96636764160,"available":96592678912,"internally_reserved":0,"allocated":16408576,"data_stored":19289695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607573,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607573,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62112,"kb_used_data":8716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309728,"statfs":{"total":96636764160,"available":96573161472,"internally_reserved":0,"allocated":8925184,"data_stored":9494402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607574,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005059,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40040,"kb_used_data":7704,"kb_used_omap":5,"kb_used_meta":32314,"kb_avail":94331800,"statfs":{"total":96636764160,"available":96595763200,"internally_reserved":0,"allocated":7888896,"data_stored":8256168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":33089829},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6508544,"data_stored":7346720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1794048,"data_stored":1745027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7573504,"data_stored":8590449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15630336,"data_stored":18970650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12410880,"data_stored":16024747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:16:13.642 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:16:13.643 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:16:13.643 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-21T23:16:13.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-21T23:16:13.661 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:13.661 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:13.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:13.728 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:13.728+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:13.812 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:13.812+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:13.873 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:13.874 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:13.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:13.936 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:13.936+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:14.013 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:14.012+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:14.114 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:14.111+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:14.136 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:14.136 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:14.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:14.195 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-21T23:16:14.192+0000 7faab395a640 -1 mon.a@0(leader).osd e69 definitely_dead 0 2024-09-21T23:16:14.237 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.1. 2024-09-21T23:16:14.237 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:14.236+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:14.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:16:14.268 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-21T23:16:14.317 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:14.311+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:14.324 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:14.324 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:14.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:14.417 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:14.416+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:14.468 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:14.468 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:14.480 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:14.524 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:14.524+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:14.619 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:14.616+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:14.640 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:14.640 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:14.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:14.740 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:14.740+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:14.807 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:14.808 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:14.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:14.822 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:14.815+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:14.922 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:14.920+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:14.988 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:14.988 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:15.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:15.044 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:15.044+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:15.123 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:15.124+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:15.164 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:15.164 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:15.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:15.224 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:15.224+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:15.247 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:16:15.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:15.325 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:15.319+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:15.354 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:15.354 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:15.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:15.425 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:15.419+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:15.500 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:15.500 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:15.514 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:15.526 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:15.524+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:15.627 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:15.624+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:15.680 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:15.680 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:15.694 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:15.727 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:15.724+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:15.809 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:15.810 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:15.824 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:15.828 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:15.828+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:15.928 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:15.923+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:15.948 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:15.949 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:15.963 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:16.050 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:16.047+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:16.130 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:16.130 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:16.130 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:16.123+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:16.144 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:16.231 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:16.228+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:16.307 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:16.307 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:16.321 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:16.370 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:16.367+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:16.399 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:16.400 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:16.430 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:16.430 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:16.433 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:16.427+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:16.458 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:16.458 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:16.479 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:16.479 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:16.487 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:16.488 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:16.495 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:16.516 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:16.516 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:16.534 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:16.532+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:16.550 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:16.550 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:16.635 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:16.632+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:16.685 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:16.686 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:16.686 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:16.686 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:16.686 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:16.702 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:16.735 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:16.732+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:16.832 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:16.832 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:16.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:16.832 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:16.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.7066443539999998, 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:16:16.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:16:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:16:16.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10124 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6156288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7022867160000001, 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038330863999999999, 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10124 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0399999999999995e-07 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125818+0000", 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125827+0000", 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6330000000000002e-06 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125834+0000", 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9199999999999998e-06 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.163896+0000", 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038062072000000002 2024-09-21T23:16:16.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164009+0000", 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011386400000000001 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164145+0000", 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013601100000000001 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10125 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6141952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7022483460000002, 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038589275999999999, 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10125 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125854+0000", 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125856+0000", 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8159999999999998e-06 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125865+0000", 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7739999999999993e-06 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125872+0000", 2024-09-21T23:16:16.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164189+0000", 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038317276999999997 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164304+0000", 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114676 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164442+0000", 2024-09-21T23:16:16.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013814 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10126 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6172672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.702220563, 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038865093000000003, 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10126 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:16.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125884+0000", 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125893+0000", 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125900+0000", 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0500000000000003e-06 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164491+0000", 2024-09-21T23:16:16.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038591473000000001 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164606+0000", 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115109 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164746+0000", 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013963800000000001 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7021931970000002, 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.853 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:16:16.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7021657239999999, 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:16:16.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:16:16.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.702125551, 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.862 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:16.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.866 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.702074782, 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:16:16.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7020441989999999, 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.871 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7020162929999998, 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.876 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:16:16.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:16:16.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.701989073, 2024-09-21T23:16:16.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:16:16.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7019621539999998, 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:16:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7019063910000001, 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.889 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:16.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:16.893 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7018732180000002, 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:16.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:16:16.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7018279989999998, 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:16.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.902 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7017887190000001, 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:16:16.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:16:16.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7017582330000001, 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.907 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:16:16.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:16.911 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7017289399999997, 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:16.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7016865980000002, 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:16:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:16:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:16:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.7013034029999998, 2024-09-21T23:16:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:16:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:16:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:16.925 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:16.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:16.927 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:16.919+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:17.026 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:17.024+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:17.057 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:17.057 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:17.057 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:17.057 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:17.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:17.160 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:17.160+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:17.228 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:17.223+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:17.242 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:17.242 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:17.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:17.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:16:17.332 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:17.327+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:17.430 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:17.428+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:17.539 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.283117201, 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:16:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:16:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.217862443, 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:16:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:16:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8205203110000001, 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:16:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8171526220000001, 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8130430420000003, 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:16:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:16:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:16:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1202 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 698368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635192+0000", 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8093089070000001, 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083075955000000007, 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1202 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635192+0000", 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635192+0000", 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635193+0000", 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635198+0000", 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5750000000000002e-06 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635218+0000", 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9729999999999999e-05 2024-09-21T23:16:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635231+0000", 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3621999999999999e-05 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.717988+0000", 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.082756266999999994 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718121+0000", 2024-09-21T23:16:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013326200000000001 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718268+0000", 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000147582 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8092501209999998, 2024-09-21T23:16:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:16:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7490913140000002, 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:16:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7453342430000003, 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7453074419999997, 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7452812770000001, 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:16:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7452528269999998, 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:16:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7451621189999997, 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:16:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7451305399999999, 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:16:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7450928499999998, 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:16:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7450624599999998, 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:16:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7450190430000001, 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:16:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:16:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7449844179999996, 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.741730918, 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:16:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7416963360000004, 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:16:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:16:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.632 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:17.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:17.633 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:17.540+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:17.639 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:17.635+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:17.740 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:17.735+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:17.750 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:16:17.794 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:17.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:17.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26591 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749401+0000", 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.021615392, 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0216364360000001, 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26591 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749401+0000", 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749401+0000", 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749402+0000", 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749403+0000", 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.249e-06 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749406+0000", 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.517e-06 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749409+0000", 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9390000000000002e-06 2024-09-21T23:16:17.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749756+0000", 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034631100000000003 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749762+0000", 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4030000000000004e-06 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754133+0000", 2024-09-21T23:16:17.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043705769999999996 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754139+0000", 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.409e-06 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26623 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.750711+0000", 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0203059919999999, 2024-09-21T23:16:17.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0204222439999999, 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26623 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750711+0000", 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750711+0000", 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750711+0000", 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5400000000000002e-07 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750714+0000", 2024-09-21T23:16:17.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4990000000000001e-06 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750722+0000", 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6640000000000004e-06 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750728+0000", 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0650000000000004e-06 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751550+0000", 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000821468 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751559+0000", 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6649999999999998e-06 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755657+0000", 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040974979999999998 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755663+0000", 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7180000000000001e-06 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26654 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.753889+0000", 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.017127372, 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017333211, 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26654 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753889+0000", 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753889+0000", 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753890+0000", 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999999e-07 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753893+0000", 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.83e-06 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753902+0000", 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4769999999999992e-06 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753908+0000", 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757129+0000", 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032209249999999999 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757137+0000", 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1659999999999994e-06 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26683 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757239+0000", 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013777964, 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0140509120000001, 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26683 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757239+0000", 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757239+0000", 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757239+0000", 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.11e-07 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757240+0000", 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.353e-06 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757251+0000", 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0353000000000001e-05 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757256+0000", 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5119999999999999e-06 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758535+0000", 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012790029999999999 2024-09-21T23:16:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758543+0000", 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3239999999999997e-06 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26713 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758491+0000", 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012525326, 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.012874896, 2024-09-21T23:16:17.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26713 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758491+0000", 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758491+0000", 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758492+0000", 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0299999999999998e-07 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758496+0000", 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4829999999999999e-06 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758509+0000", 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3573e-05 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758516+0000", 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6919999999999996e-06 2024-09-21T23:16:17.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759868+0000", 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013520870000000001 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759871+0000", 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9110000000000002e-06 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27769 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759583+0000", 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011433869, 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0118665760000001, 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27769 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759583+0000", 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759583+0000", 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759583+0000", 2024-09-21T23:16:17.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8899999999999999e-07 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759587+0000", 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5710000000000002e-06 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759596+0000", 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1889999999999996e-06 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759603+0000", 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3539999999999996e-06 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760771+0000", 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001168596 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760777+0000", 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27801 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760615+0000", 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010401954, 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0108987679999999, 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27801 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760615+0000", 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760615+0000", 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760615+0000", 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760617+0000", 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1940000000000001e-06 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760624+0000", 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0779999999999998e-06 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760630+0000", 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3859999999999998e-06 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761593+0000", 2024-09-21T23:16:17.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096310700000000005 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761599+0000", 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009296151, 2024-09-21T23:16:17.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.009859157, 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.829 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26592 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749450+0000", 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0215670880000001, 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0221935069999999, 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26592 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749450+0000", 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749450+0000", 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749451+0000", 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.083e-06 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749455+0000", 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7650000000000001e-06 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749466+0000", 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0983e-05 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749477+0000", 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0463e-05 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749804+0000", 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032667699999999999 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749809+0000", 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3149999999999999e-06 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754167+0000", 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043579710000000004 2024-09-21T23:16:17.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754173+0000", 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2480000000000001e-06 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26624 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.750736+0000", 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.02028121, 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0209824279999999, 2024-09-21T23:16:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26624 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750736+0000", 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750736+0000", 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750736+0000", 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5699999999999998e-07 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750738+0000", 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1100000000000001e-06 2024-09-21T23:16:17.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750775+0000", 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6795000000000003e-05 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750787+0000", 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2493e-05 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751590+0000", 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080258599999999999 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751598+0000", 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2320000000000001e-06 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755686+0000", 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040880830000000002 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755692+0000", 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26655 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.753926+0000", 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.017090804, 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017865614, 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26655 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753926+0000", 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753926+0000", 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753926+0000", 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6899999999999998e-07 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753929+0000", 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5119999999999998e-06 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753936+0000", 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4259999999999997e-06 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753941+0000", 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0590000000000002e-06 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757194+0000", 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032526109999999999 2024-09-21T23:16:17.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757202+0000", 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.028e-06 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26684 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757283+0000", 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.01373333, 2024-09-21T23:16:17.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0145705570000001, 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26684 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757283+0000", 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757283+0000", 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757284+0000", 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757287+0000", 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-21T23:16:17.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757300+0000", 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2864000000000001e-05 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757304+0000", 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0609999999999997e-06 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758580+0000", 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001275918 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758587+0000", 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26714 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.847 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758527+0000", 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0124897390000001, 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.013392077, 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26714 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758527+0000", 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758527+0000", 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758527+0000", 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758530+0000", 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.864e-06 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758538+0000", 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8460000000000008e-06 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758544+0000", 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759980+0000", 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014367169999999999 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759988+0000", 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1409999999999999e-06 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27770 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759621+0000", 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011395673, 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0123660269999999, 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27770 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759621+0000", 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759621+0000", 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759622+0000", 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759624+0000", 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9229999999999998e-06 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759642+0000", 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7294000000000001e-05 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759649+0000", 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0129999999999996e-06 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760807+0000", 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011578409999999999 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760813+0000", 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2169999999999999e-06 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27802 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760637+0000", 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0103793889999999, 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.011414126, 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27802 2024-09-21T23:16:17.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760637+0000", 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760637+0000", 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760638+0000", 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7800000000000002e-07 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760640+0000", 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760647+0000", 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8589999999999999e-06 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760652+0000", 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.181e-06 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761632+0000", 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00097969300000000001 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761638+0000", 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9780000000000002e-06 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009266456, 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0103631049999999, 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:17.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26593 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749673+0000", 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.021343758, 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0225054330000001, 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26593 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749673+0000", 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749673+0000", 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749674+0000", 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2600000000000002e-07 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749677+0000", 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8550000000000004e-06 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749686+0000", 2024-09-21T23:16:17.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749692+0000", 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3559999999999998e-06 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749845+0000", 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000152671 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749850+0000", 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3970000000000002e-06 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754198+0000", 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043479599999999997 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754205+0000", 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26625 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.750833+0000", 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.020183785, 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021415339, 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26625 2024-09-21T23:16:17.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750833+0000", 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750833+0000", 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750834+0000", 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2699999999999998e-07 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750838+0000", 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750849+0000", 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0592e-05 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750856+0000", 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751711+0000", 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085482199999999998 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751714+0000", 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755715+0000", 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040001890000000003 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755720+0000", 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4410000000000001e-06 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.873 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26685 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757347+0000", 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0136698710000001, 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.014973715, 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26685 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757347+0000", 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757347+0000", 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757347+0000", 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757350+0000", 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757358+0000", 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2600000000000005e-06 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757363+0000", 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4859999999999997e-06 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758615+0000", 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001251114 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758621+0000", 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1909999999999997e-06 2024-09-21T23:16:17.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26715 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758553+0000", 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0124640309999999, 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.013833049, 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26715 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758553+0000", 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758553+0000", 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758553+0000", 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9200000000000001e-07 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758556+0000", 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6579999999999998e-06 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758564+0000", 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5560000000000002e-06 2024-09-21T23:16:17.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758569+0000", 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6189999999999999e-06 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760020+0000", 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014509220000000001 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760026+0000", 2024-09-21T23:16:17.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3330000000000002e-06 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27771 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759678+0000", 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011338638, 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.012767229, 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27771 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759678+0000", 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759678+0000", 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759679+0000", 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6300000000000005e-07 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759682+0000", 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4209999999999999e-06 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759691+0000", 2024-09-21T23:16:17.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0089999999999996e-06 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759697+0000", 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8359999999999997e-06 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760841+0000", 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001144452 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760847+0000", 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8810000000000001e-06 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27803 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760667+0000", 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0103494230000001, 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0118408619999999, 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27803 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760667+0000", 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760667+0000", 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760668+0000", 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760670+0000", 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7010000000000001e-06 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760678+0000", 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760683+0000", 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761666+0000", 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098291799999999994 2024-09-21T23:16:17.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761670+0000", 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.49e-06 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009233528, 2024-09-21T23:16:17.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0107964700000001, 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:17.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26594 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749703+0000", 2024-09-21T23:16:17.894 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0213133679999999, 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.022953947, 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26594 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749703+0000", 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749703+0000", 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749704+0000", 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.08e-07 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749706+0000", 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5179999999999999e-06 2024-09-21T23:16:17.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749714+0000", 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6720000000000004e-06 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749720+0000", 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7119999999999997e-06 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749878+0000", 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158059 2024-09-21T23:16:17.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749883+0000", 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3079999999999998e-06 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754232+0000", 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043485800000000003 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754238+0000", 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7220000000000002e-06 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26626 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.750866+0000", 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.020150949, 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0218831749999999, 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26626 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750866+0000", 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750866+0000", 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750866+0000", 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750869+0000", 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7369999999999998e-06 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750876+0000", 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0820000000000001e-06 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750881+0000", 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1569999999999999e-06 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751887+0000", 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001005927 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751891+0000", 2024-09-21T23:16:17.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5039999999999998e-06 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755744+0000", 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038535710000000001 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755750+0000", 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3329999999999996e-06 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26656 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754097+0000", 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0169198290000001, 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0187365269999999, 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26656 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754097+0000", 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754097+0000", 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754097+0000", 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6400000000000002e-07 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754114+0000", 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6299000000000001e-05 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754127+0000", 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3380000000000001e-05 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754134+0000", 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757247+0000", 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031121930000000001 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757254+0000", 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5490000000000001e-06 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26686 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757387+0000", 2024-09-21T23:16:17.907 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013629629, 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015519227, 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26686 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757387+0000", 2024-09-21T23:16:17.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757387+0000", 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757388+0000", 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1899999999999998e-07 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757390+0000", 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8210000000000002e-06 2024-09-21T23:16:17.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757398+0000", 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1189999999999997e-06 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757406+0000", 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0899999999999999e-06 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758654+0000", 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001248646 2024-09-21T23:16:17.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758658+0000", 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4860000000000002e-06 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26716 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758586+0000", 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012430567, 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0144444020000001, 2024-09-21T23:16:17.911 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26716 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758586+0000", 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758586+0000", 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758587+0000", 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7000000000000005e-07 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758590+0000", 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8109999999999999e-06 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758597+0000", 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5989999999999996e-06 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758603+0000", 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8000000000000004e-06 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760054+0000", 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014507350000000001 2024-09-21T23:16:17.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760059+0000", 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4790000000000004e-06 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27772 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759719+0000", 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0112980840000001, 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0134091590000001, 2024-09-21T23:16:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27772 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759719+0000", 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759719+0000", 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759719+0000", 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1399999999999997e-07 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759722+0000", 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.914e-06 2024-09-21T23:16:17.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759730+0000", 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759736+0000", 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7479999999999999e-06 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760876+0000", 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001139791 2024-09-21T23:16:17.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760882+0000", 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1809999999999998e-06 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27804 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760699+0000", 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0103176250000001, 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0125216189999999, 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27804 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760699+0000", 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760699+0000", 2024-09-21T23:16:17.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760700+0000", 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.34e-07 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760702+0000", 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.689e-06 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760711+0000", 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1579999999999994e-06 2024-09-21T23:16:17.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760717+0000", 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7930000000000003e-06 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761684+0000", 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096646700000000004 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761686+0000", 2024-09-21T23:16:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0091992320000001, 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.011482021, 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:17.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26595 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749728+0000", 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.02128873, 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023644512, 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26595 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749728+0000", 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749728+0000", 2024-09-21T23:16:17.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749728+0000", 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4400000000000001e-07 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749731+0000", 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7089999999999998e-06 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749738+0000", 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0670000000000003e-06 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749743+0000", 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3510000000000001e-06 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749920+0000", 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000176064 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749925+0000", 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6579999999999999e-06 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754267+0000", 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043417309999999997 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754273+0000", 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2489999999999998e-06 2024-09-21T23:16:17.931 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26657 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754145+0000", 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0168721519999999, 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019305269, 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.932 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26657 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754145+0000", 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754145+0000", 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754145+0000", 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.84e-07 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754150+0000", 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1460000000000004e-06 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754155+0000", 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7119999999999999e-06 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754158+0000", 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2650000000000001e-06 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757288+0000", 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031298939999999998 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757297+0000", 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3670000000000003e-06 2024-09-21T23:16:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26687 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757415+0000", 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0136021180000001, 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016104546, 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.936 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26687 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757415+0000", 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757415+0000", 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757415+0000", 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.01e-07 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757417+0000", 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-06 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757426+0000", 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3729999999999993e-06 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757431+0000", 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7740000000000001e-06 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758691+0000", 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001259661 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758697+0000", 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4110000000000001e-06 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.939 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26717 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758613+0000", 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0124032380000001, 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.014970712, 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26717 2024-09-21T23:16:17.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758613+0000", 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758613+0000", 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758614+0000", 2024-09-21T23:16:17.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758617+0000", 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758624+0000", 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6909999999999997e-06 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758632+0000", 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5059999999999998e-06 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760087+0000", 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001455385 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760091+0000", 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7919999999999999e-06 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27773 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759757+0000", 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011260139, 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.013896318, 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27773 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759757+0000", 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759757+0000", 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759757+0000", 2024-09-21T23:16:17.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4099999999999999e-07 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759760+0000", 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.151e-06 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759768+0000", 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.79e-06 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759775+0000", 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7909999999999999e-06 2024-09-21T23:16:17.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760910+0000", 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011351340000000001 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760916+0000", 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.947 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27805 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760737+0000", 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010279755, 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0129812659999999, 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27805 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760737+0000", 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760737+0000", 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760737+0000", 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9900000000000001e-07 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760740+0000", 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.779e-06 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760748+0000", 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7759999999999997e-06 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760755+0000", 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2119999999999997e-06 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761700+0000", 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094490700000000004 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761703+0000", 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8399999999999999e-06 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:17.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0091747259999999, 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.011946443, 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:17.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:17.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26596 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749751+0000", 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.021265283, 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0241005839999999, 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26596 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749751+0000", 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749751+0000", 2024-09-21T23:16:17.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749752+0000", 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4499999999999998e-07 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749754+0000", 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3920000000000001e-06 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749761+0000", 2024-09-21T23:16:17.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8920000000000003e-06 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749766+0000", 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1070000000000004e-06 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749979+0000", 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021288899999999999 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749987+0000", 2024-09-21T23:16:17.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3780000000000001e-06 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754301+0000", 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043134250000000001 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754307+0000", 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.960 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26627 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.960 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.751859+0000", 2024-09-21T23:16:17.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0191580389999999, 2024-09-21T23:16:17.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.02207227, 2024-09-21T23:16:17.961 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.961 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.961 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26627 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751859+0000", 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751859+0000", 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751859+0000", 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2099999999999996e-07 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751867+0000", 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2239999999999998e-06 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751876+0000", 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8309999999999996e-06 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.751884+0000", 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4649999999999997e-06 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755774+0000", 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038902329999999999 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755780+0000", 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8719999999999999e-06 2024-09-21T23:16:17.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26658 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754162+0000", 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0168542860000001, 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0198391769999999, 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26658 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754162+0000", 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754162+0000", 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754163+0000", 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3099999999999999e-07 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754164+0000", 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2389999999999999e-06 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754183+0000", 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8635999999999999e-05 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754198+0000", 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5909999999999998e-05 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757355+0000", 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031568260000000002 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757363+0000", 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26688 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757451+0000", 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0135655729999999, 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166163079999999, 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26688 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757451+0000", 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757451+0000", 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757452+0000", 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7e-07 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757455+0000", 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.022e-06 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757462+0000", 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7729999999999999e-06 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757468+0000", 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8669999999999999e-06 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758732+0000", 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012632699999999999 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758735+0000", 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5820000000000002e-06 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26718 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758649+0000", 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012367526, 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0154854630000001, 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26718 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758649+0000", 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758649+0000", 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758650+0000", 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-21T23:16:17.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758653+0000", 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0589999999999998e-06 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758661+0000", 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2570000000000007e-06 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758667+0000", 2024-09-21T23:16:17.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760133+0000", 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001466137 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760140+0000", 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3570000000000003e-06 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27774 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759785+0000", 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.01123205, 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.014412141, 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27774 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759785+0000", 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759785+0000", 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759785+0000", 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759788+0000", 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7429999999999998e-06 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759796+0000", 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6399999999999997e-06 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759801+0000", 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.806e-06 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760951+0000", 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001150105 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760957+0000", 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9920000000000004e-06 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27806 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760764+0000", 2024-09-21T23:16:17.980 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0102523400000001, 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.013502482, 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27806 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760764+0000", 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760764+0000", 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760765+0000", 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7300000000000002e-07 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760767+0000", 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4449999999999999e-06 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760775+0000", 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3189999999999997e-06 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760780+0000", 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4439999999999999e-06 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761718+0000", 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000938473 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6919999999999998e-06 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0091467359999999, 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.012468393, 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.984 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:17.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:17.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26597 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749819+0000", 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0211975010000001, 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0245841099999999, 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26597 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749819+0000", 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749819+0000", 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749820+0000", 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749823+0000", 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0469999999999998e-06 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749830+0000", 2024-09-21T23:16:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6879999999999999e-06 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749836+0000", 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8069999999999996e-06 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750035+0000", 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019868300000000001 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750041+0000", 2024-09-21T23:16:17.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3999999999999997e-06 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754338+0000", 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042962290000000004 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754344+0000", 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26628 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752091+0000", 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018925292, 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.022392886, 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26628 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.993 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752091+0000", 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752091+0000", 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752092+0000", 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8599999999999996e-07 2024-09-21T23:16:17.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752102+0000", 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5040000000000008e-06 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752113+0000", 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1209000000000001e-05 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752121+0000", 2024-09-21T23:16:17.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8930000000000005e-06 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755806+0000", 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003684863 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755812+0000", 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7830000000000004e-06 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26659 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754208+0000", 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0168091130000001, 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.020343505, 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26659 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:17.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754208+0000", 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754208+0000", 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754208+0000", 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0100000000000001e-07 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:17.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754210+0000", 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3020000000000002e-06 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754215+0000", 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4259999999999996e-06 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754218+0000", 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6710000000000001e-06 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:17.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757409+0000", 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031909769999999998 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757417+0000", 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2980000000000003e-06 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26689 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757497+0000", 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0135201170000001, 2024-09-21T23:16:18.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0171244740000001, 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26689 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757497+0000", 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757497+0000", 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757497+0000", 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5800000000000001e-07 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757499+0000", 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.996e-06 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757506+0000", 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2110000000000001e-06 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757512+0000", 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5289999999999999e-06 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758753+0000", 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001241081 2024-09-21T23:16:18.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758758+0000", 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7930000000000003e-06 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26719 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758683+0000", 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012334214, 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160228529999999, 2024-09-21T23:16:18.004 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26719 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758683+0000", 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758683+0000", 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758683+0000", 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1099999999999996e-07 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758686+0000", 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9100000000000001e-06 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758694+0000", 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7919999999999993e-06 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758701+0000", 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760173+0000", 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014727399999999999 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760180+0000", 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3960000000000003e-06 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27775 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759818+0000", 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011198563, 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.014954897, 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27775 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759818+0000", 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759818+0000", 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759819+0000", 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759828+0000", 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2029999999999998e-06 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759836+0000", 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6850000000000007e-06 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759843+0000", 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8539999999999999e-06 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760986+0000", 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00114284 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760992+0000", 2024-09-21T23:16:18.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8529999999999997e-06 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27807 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760802+0000", 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0102148740000001, 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0140423599999999, 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27807 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760802+0000", 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760802+0000", 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760802+0000", 2024-09-21T23:16:18.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2500000000000001e-07 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760805+0000", 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760813+0000", 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4440000000000008e-06 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760821+0000", 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091319800000000005 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761737+0000", 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9210000000000001e-06 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009123708, 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0130181380000001, 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:18.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26598 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749846+0000", 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0211711910000001, 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0252295920000001, 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26598 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749846+0000", 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749846+0000", 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749846+0000", 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749854+0000", 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8690000000000004e-06 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749860+0000", 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.353e-06 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749865+0000", 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0379999999999999e-06 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750083+0000", 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000217703 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750088+0000", 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4369999999999998e-06 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754373+0000", 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004284955 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754424+0000", 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1057000000000003e-05 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26629 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752135+0000", 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0188821189999999, 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023022221, 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26629 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752135+0000", 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752135+0000", 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752135+0000", 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752138+0000", 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9160000000000001e-06 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752145+0000", 2024-09-21T23:16:18.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2849999999999997e-06 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752153+0000", 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5220000000000001e-06 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755836+0000", 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036830819999999998 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755841+0000", 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5829999999999997e-06 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26660 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754225+0000", 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0167919059999999, 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0209937629999999, 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26660 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754225+0000", 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754225+0000", 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754225+0000", 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2099999999999998e-07 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754228+0000", 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4219999999999999e-06 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754234+0000", 2024-09-21T23:16:18.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6680000000000004e-06 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754239+0000", 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1789999999999999e-06 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757457+0000", 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00321792 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757464+0000", 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9280000000000004e-06 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26690 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757520+0000", 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013496363, 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017770872, 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26690 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757520+0000", 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757520+0000", 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757521+0000", 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999999e-07 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757523+0000", 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.553e-06 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757531+0000", 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.435e-06 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757536+0000", 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758789+0000", 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001252228 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758812+0000", 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3591000000000002e-05 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26720 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758726+0000", 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0122903329999999, 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166465790000001, 2024-09-21T23:16:18.036 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26720 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758726+0000", 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758726+0000", 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758727+0000", 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758730+0000", 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2260000000000001e-06 2024-09-21T23:16:18.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758738+0000", 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1550000000000007e-06 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758744+0000", 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0059999999999998e-06 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760206+0000", 2024-09-21T23:16:18.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001462259 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760213+0000", 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2380000000000002e-06 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27776 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759854+0000", 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011163118, 2024-09-21T23:16:18.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015604416, 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27776 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759854+0000", 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759854+0000", 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759854+0000", 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.13e-07 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759857+0000", 2024-09-21T23:16:18.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.79e-06 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759864+0000", 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2490000000000004e-06 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759870+0000", 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4240000000000001e-06 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761025+0000", 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011558580000000001 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761031+0000", 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8250000000000001e-06 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27808 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760837+0000", 2024-09-21T23:16:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0101796160000001, 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.014698447, 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27808 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760837+0000", 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760837+0000", 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760838+0000", 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9900000000000001e-07 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760841+0000", 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4580000000000002e-06 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760849+0000", 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8490000000000006e-06 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760855+0000", 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-21T23:16:18.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089489599999999995 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761752+0000", 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8420000000000001e-06 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.048 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0091021979999999, 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.013690194, 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:18.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26599 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749872+0000", 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0211447440000001, 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0258091220000001, 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26599 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749872+0000", 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749872+0000", 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749872+0000", 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5100000000000001e-07 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749874+0000", 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.894e-06 2024-09-21T23:16:18.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749880+0000", 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1829999999999999e-06 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749886+0000", 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4929999999999998e-06 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750143+0000", 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025745599999999998 2024-09-21T23:16:18.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750149+0000", 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2440000000000001e-06 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754463+0000", 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043143249999999999 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754470+0000", 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-09-21T23:16:18.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26630 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752161+0000", 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0188556689999999, 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023599814, 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26630 2024-09-21T23:16:18.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752161+0000", 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752161+0000", 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752161+0000", 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.84e-07 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752164+0000", 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2970000000000002e-06 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752170+0000", 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1650000000000003e-06 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752176+0000", 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2149999999999998e-06 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755869+0000", 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036932530000000001 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755875+0000", 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6459999999999998e-06 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26661 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754262+0000", 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.016754465, 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021570179, 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26661 2024-09-21T23:16:18.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754262+0000", 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754262+0000", 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754263+0000", 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754265+0000", 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6259999999999999e-06 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754273+0000", 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0749999999999998e-06 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754279+0000", 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.288e-06 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757506+0000", 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003227026 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757513+0000", 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26722 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758753+0000", 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0122634559999999, 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017151892, 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26722 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758753+0000", 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758753+0000", 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758754+0000", 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.03e-07 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758756+0000", 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.463e-06 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758763+0000", 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0820000000000001e-06 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758768+0000", 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-21T23:16:18.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760239+0000", 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001470832 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760245+0000", 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0909999999999998e-06 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.068 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27777 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759884+0000", 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011132712, 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160924259999999, 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27777 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759884+0000", 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759884+0000", 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759885+0000", 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759888+0000", 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9790000000000002e-06 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759895+0000", 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6070000000000002e-06 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759901+0000", 2024-09-21T23:16:18.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6749999999999999e-06 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761060+0000", 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001159425 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761068+0000", 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1130000000000001e-06 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27809 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760871+0000", 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010145879, 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0151813080000001, 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27809 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760871+0000", 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760871+0000", 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760871+0000", 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3300000000000003e-07 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760874+0000", 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0450000000000001e-06 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760882+0000", 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7360000000000001e-06 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760888+0000", 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.772e-06 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761765+0000", 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087684600000000003 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761768+0000", 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.942e-06 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.076 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009066657, 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0141712110000001, 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26600 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.749919+0000", 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0210981269999999, 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026276027, 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26600 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749919+0000", 2024-09-21T23:16:18.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749919+0000", 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749919+0000", 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7300000000000005e-07 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749924+0000", 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749966+0000", 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.749992+0000", 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5874e-05 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750770+0000", 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077767199999999998 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750777+0000", 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7130000000000005e-06 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754508+0000", 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003730475 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754514+0000", 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6719999999999998e-06 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26631 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752183+0000", 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018833393, 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0241003829999999, 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26631 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752183+0000", 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752183+0000", 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752184+0000", 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752186+0000", 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752192+0000", 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.533e-06 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752198+0000", 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755900+0000", 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003701878 2024-09-21T23:16:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755906+0000", 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1079999999999998e-06 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26662 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754297+0000", 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.016719487, 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.022058379, 2024-09-21T23:16:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26662 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754297+0000", 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754297+0000", 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754298+0000", 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2500000000000001e-07 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754300+0000", 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4710000000000001e-06 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754307+0000", 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754312+0000", 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8199999999999996e-06 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757558+0000", 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032458270000000002 2024-09-21T23:16:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757570+0000", 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1646e-05 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26691 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757555+0000", 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0134616219999999, 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0188770739999999, 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26691 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757555+0000", 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757555+0000", 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757556+0000", 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7800000000000002e-07 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757559+0000", 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757567+0000", 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4880000000000007e-06 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757573+0000", 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4410000000000001e-06 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758893+0000", 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001320662 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758902+0000", 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0219999999999993e-06 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26724 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758784+0000", 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0122325599999999, 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0177174010000001, 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26724 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758784+0000", 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758784+0000", 2024-09-21T23:16:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758785+0000", 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758787+0000", 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8329999999999998e-06 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758802+0000", 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4063000000000001e-05 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758810+0000", 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1659999999999994e-06 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760270+0000", 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001460757 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760276+0000", 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9150000000000001e-06 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27778 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759909+0000", 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011107288, 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166677879999999, 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27778 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759909+0000", 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759909+0000", 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759910+0000", 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8000000000000001e-07 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759912+0000", 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.277e-06 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759919+0000", 2024-09-21T23:16:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999998e-06 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759924+0000", 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1200000000000001e-06 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761090+0000", 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011665289999999999 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761094+0000", 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2320000000000001e-06 2024-09-21T23:16:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27810 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760909+0000", 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0101080019999999, 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015752465, 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27810 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760909+0000", 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760909+0000", 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760909+0000", 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4299999999999998e-07 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760912+0000", 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8090000000000001e-06 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760920+0000", 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760926+0000", 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3559999999999998e-06 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761781+0000", 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085532800000000003 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.892e-06 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0090390199999999, 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0147575310000001, 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26601 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.750019+0000", 2024-09-21T23:16:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.020998214, 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0268009259999999, 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26601 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750019+0000", 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750019+0000", 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750019+0000", 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3600000000000003e-07 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750032+0000", 2024-09-21T23:16:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2353e-05 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750059+0000", 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7787e-05 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750077+0000", 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7887e-05 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750837+0000", 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075989199999999997 2024-09-21T23:16:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.750842+0000", 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3349999999999997e-06 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754574+0000", 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037316810000000001 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754580+0000", 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-21T23:16:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26632 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752205+0000", 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0188116149999999, 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0247044919999999, 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26632 2024-09-21T23:16:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752205+0000", 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752205+0000", 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752205+0000", 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-07 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752216+0000", 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.008e-05 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752223+0000", 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752228+0000", 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0560000000000004e-06 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755932+0000", 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037042120000000001 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755938+0000", 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.037e-06 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26663 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754342+0000", 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.016674436, 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0226577480000001, 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26663 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754342+0000", 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754342+0000", 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754343+0000", 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754346+0000", 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9730000000000002e-06 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754355+0000", 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8400000000000001e-06 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754362+0000", 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.029e-06 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757608+0000", 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032467580000000002 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757615+0000", 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5119999999999997e-06 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26692 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757603+0000", 2024-09-21T23:16:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0134142129999999, 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0194821570000001, 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26692 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757603+0000", 2024-09-21T23:16:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757603+0000", 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757603+0000", 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5199999999999998e-07 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757617+0000", 2024-09-21T23:16:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4243000000000001e-05 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757623+0000", 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2589999999999997e-06 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757627+0000", 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8449999999999996e-06 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758956+0000", 2024-09-21T23:16:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013288449999999999 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758965+0000", 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26726 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758819+0000", 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012197848, 2024-09-21T23:16:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018344388, 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26726 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758819+0000", 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758819+0000", 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758819+0000", 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758822+0000", 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9670000000000001e-06 2024-09-21T23:16:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758831+0000", 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7709999999999995e-06 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758837+0000", 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6690000000000003e-06 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760306+0000", 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014694580000000001 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760312+0000", 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9009999999999999e-06 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27779 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759932+0000", 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011084747, 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0173122000000001, 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27779 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759932+0000", 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759932+0000", 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759932+0000", 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0900000000000002e-07 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759937+0000", 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.143e-06 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759944+0000", 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1180000000000003e-06 2024-09-21T23:16:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759951+0000", 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0369999999999997e-06 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761108+0000", 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001157496 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761111+0000", 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.903e-06 2024-09-21T23:16:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27811 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760946+0000", 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0100705940000001, 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0163729100000001, 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27811 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760946+0000", 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760946+0000", 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760947+0000", 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.89e-07 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760956+0000", 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5219999999999996e-06 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760965+0000", 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9060000000000008e-06 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760971+0000", 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761797+0000", 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082583199999999998 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761800+0000", 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.762e-06 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0090038560000001, 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.01538599, 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26633 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752236+0000", 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0187807470000001, 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025246654, 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26633 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752236+0000", 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752236+0000", 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752236+0000", 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752239+0000", 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2180000000000002e-06 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752245+0000", 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5030000000000002e-06 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752250+0000", 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2179999999999998e-06 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755961+0000", 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003710652 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755967+0000", 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6189999999999999e-06 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26664 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754371+0000", 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0166453579999999, 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0232068379999999, 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26664 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754371+0000", 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754371+0000", 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754372+0000", 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6199999999999999e-07 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754374+0000", 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4250000000000001e-06 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754381+0000", 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1239999999999999e-06 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754387+0000", 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2399999999999998e-06 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757644+0000", 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032578910000000002 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757651+0000", 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26693 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757639+0000", 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0133775890000001, 2024-09-21T23:16:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.020013337, 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26693 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757639+0000", 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757639+0000", 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757639+0000", 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8799999999999998e-07 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757641+0000", 2024-09-21T23:16:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6729999999999999e-06 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757645+0000", 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757648+0000", 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0450000000000001e-06 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759006+0000", 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001357607 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759013+0000", 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6019999999999996e-06 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26728 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758859+0000", 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012158127, 2024-09-21T23:16:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0188718000000001, 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26728 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758859+0000", 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758859+0000", 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758859+0000", 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758862+0000", 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7709999999999998e-06 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758870+0000", 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1119999999999996e-06 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758876+0000", 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8719999999999999e-06 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760340+0000", 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014639939999999999 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760346+0000", 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0070000000000003e-06 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27780 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759959+0000", 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011058142, 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178520719999999, 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27780 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759959+0000", 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759959+0000", 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759959+0000", 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759962+0000", 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6350000000000002e-06 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759969+0000", 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0550000000000002e-06 2024-09-21T23:16:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759974+0000", 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6740000000000002e-06 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761124+0000", 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001149312 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761126+0000", 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.813e-06 2024-09-21T23:16:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27812 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760980+0000", 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010037004, 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0169020179999999, 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27812 2024-09-21T23:16:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760980+0000", 2024-09-21T23:16:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760980+0000", 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760980+0000", 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760982+0000", 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3259999999999998e-06 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760990+0000", 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1960000000000002e-06 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760995+0000", 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3290000000000001e-06 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761812+0000", 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081700900000000005 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761815+0000", 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8760000000000001e-06 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.008968825, 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015915546, 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26634 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752257+0000", 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0187597049999999, 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.02579676, 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26634 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752257+0000", 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752257+0000", 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752257+0000", 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1699999999999999e-07 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752260+0000", 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3300000000000001e-06 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752266+0000", 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2609999999999999e-06 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752273+0000", 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2060000000000001e-06 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756340+0000", 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040663679999999999 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756347+0000", 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8159999999999994e-06 2024-09-21T23:16:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26665 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754395+0000", 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0166220669999999, 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023733094, 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26665 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754395+0000", 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754395+0000", 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754395+0000", 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.08e-07 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754397+0000", 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.198e-06 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754404+0000", 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7710000000000001e-06 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754410+0000", 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9699999999999996e-06 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757680+0000", 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032695609999999998 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757698+0000", 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8567e-05 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26694 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757653+0000", 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013363859, 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.020545528, 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26694 2024-09-21T23:16:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757653+0000", 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757653+0000", 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757653+0000", 2024-09-21T23:16:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.28e-07 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757654+0000", 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3149999999999999e-06 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757658+0000", 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3519999999999998e-06 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757661+0000", 2024-09-21T23:16:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8839999999999998e-06 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759323+0000", 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016623860000000001 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759331+0000", 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0460000000000006e-06 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26730 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758884+0000", 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0121322669999999, 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019382553, 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26730 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758884+0000", 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758884+0000", 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758885+0000", 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758887+0000", 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2450000000000001e-06 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758896+0000", 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8079999999999994e-06 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758901+0000", 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3589999999999999e-06 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760373+0000", 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001471775 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760379+0000", 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8830000000000003e-06 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27781 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759983+0000", 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0110341350000001, 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0183568599999999, 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27781 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759983+0000", 2024-09-21T23:16:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759983+0000", 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759983+0000", 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5400000000000002e-07 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759986+0000", 2024-09-21T23:16:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5840000000000002e-06 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759992+0000", 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8290000000000002e-06 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759998+0000", 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3890000000000004e-06 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761140+0000", 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011423869999999999 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761143+0000", 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3500000000000001e-06 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27813 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761020+0000", 2024-09-21T23:16:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0099962330000001, 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0173843970000001, 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27813 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761020+0000", 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761020+0000", 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761021+0000", 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761024+0000", 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0419999999999998e-06 2024-09-21T23:16:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761033+0000", 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5650000000000002e-06 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761039+0000", 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0499999999999997e-06 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761828+0000", 2024-09-21T23:16:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078957500000000004 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761831+0000", 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.976e-06 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0089334400000001, 2024-09-21T23:16:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0163934080000001, 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27862 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762704+0000", 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.008312812, 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0158399149999999, 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27862 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762704+0000", 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762704+0000", 2024-09-21T23:16:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770911+0000", 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0082072900000001 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770920+0000", 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4020000000000002e-06 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770952+0000", 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2858999999999998e-05 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770963+0000", 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0961999999999999e-05 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26635 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752281+0000", 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0187358470000001, 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0263229700000001, 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26635 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752281+0000", 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752281+0000", 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752281+0000", 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7599999999999998e-07 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752283+0000", 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1579999999999999e-06 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752290+0000", 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.657e-06 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752296+0000", 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6370000000000004e-06 2024-09-21T23:16:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756376+0000", 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040805160000000002 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756382+0000", 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26666 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754418+0000", 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0165992230000001, 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0242521760000001, 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26666 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754418+0000", 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754418+0000", 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754418+0000", 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.03e-07 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754420+0000", 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.057e-06 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754427+0000", 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7039999999999997e-06 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754432+0000", 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4410000000000001e-06 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757758+0000", 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033255139999999999 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757765+0000", 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8320000000000006e-06 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26695 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757676+0000", 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0133412180000001, 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021073745, 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26695 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757676+0000", 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757676+0000", 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757676+0000", 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757678+0000", 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5720000000000001e-06 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757692+0000", 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.346e-05 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757698+0000", 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4239999999999998e-06 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759364+0000", 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001665719 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759371+0000", 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3780000000000003e-06 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26732 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758910+0000", 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012106709, 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.01990994, 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26732 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758910+0000", 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758910+0000", 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758910+0000", 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758913+0000", 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3630000000000001e-06 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758920+0000", 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9090000000000003e-06 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758925+0000", 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2120000000000002e-06 2024-09-21T23:16:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760409+0000", 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014841349999999999 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760415+0000", 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9139999999999996e-06 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27782 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760018+0000", 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0109985420000001, 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0188690380000001, 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27782 2024-09-21T23:16:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760018+0000", 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760018+0000", 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760019+0000", 2024-09-21T23:16:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6400000000000003e-07 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760022+0000", 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.881e-06 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760030+0000", 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760036+0000", 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8150000000000002e-06 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761158+0000", 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011226929999999999 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761161+0000", 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.959e-06 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27814 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761056+0000", 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0099603589999999, 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0179007579999999, 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27814 2024-09-21T23:16:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761056+0000", 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761056+0000", 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761057+0000", 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4700000000000002e-07 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761059+0000", 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5390000000000002e-06 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761069+0000", 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.533e-06 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761075+0000", 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0449999999999997e-06 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761844+0000", 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076860600000000002 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761846+0000", 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9469999999999999e-06 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0088932580000001, 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168995700000001, 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27863 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.770981+0000", 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5349000000000003e-05, 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0081149680000000002, 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27863 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770981+0000", 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770981+0000", 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770982+0000", 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0699999999999997e-07 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770985+0000", 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2490000000000001e-06 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770993+0000", 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.559e-06 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.770997+0000", 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.34e-06 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26636 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752302+0000", 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018714283, 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026858327, 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26636 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752302+0000", 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752302+0000", 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752303+0000", 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.46e-07 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752305+0000", 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.08e-06 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752311+0000", 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9669999999999998e-06 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752316+0000", 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9960000000000001e-06 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756411+0000", 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040953300000000003 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756417+0000", 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26667 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754457+0000", 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0165599910000001, 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0248066730000001, 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26667 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754457+0000", 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754457+0000", 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754457+0000", 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754460+0000", 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5940000000000001e-06 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754468+0000", 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0930000000000003e-06 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754473+0000", 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4380000000000003e-06 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757819+0000", 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033452429999999999 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757829+0000", 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0856000000000001e-05 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26696 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757707+0000", 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0133092859999999, 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0216241269999999, 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26696 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757707+0000", 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757707+0000", 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757708+0000", 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999999e-07 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757711+0000", 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7190000000000001e-06 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757732+0000", 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1857000000000001e-05 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757739+0000", 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.968e-06 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759401+0000", 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001661556 2024-09-21T23:16:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759407+0000", 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0859999999999998e-06 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26734 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758949+0000", 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012067754, 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.020449288, 2024-09-21T23:16:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26734 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758949+0000", 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758949+0000", 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758950+0000", 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5199999999999995e-07 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758961+0000", 2024-09-21T23:16:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0987e-05 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758971+0000", 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0373e-05 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758979+0000", 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6070000000000002e-06 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760444+0000", 2024-09-21T23:16:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001464985 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760454+0000", 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0691e-05 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27783 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760047+0000", 2024-09-21T23:16:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0109698359999999, 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019419345, 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27783 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760047+0000", 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760047+0000", 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760047+0000", 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2100000000000002e-07 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760050+0000", 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.666e-06 2024-09-21T23:16:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760058+0000", 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3720000000000005e-06 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760065+0000", 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1530000000000002e-06 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761177+0000", 2024-09-21T23:16:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011120080000000001 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761179+0000", 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6910000000000002e-06 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27815 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761091+0000", 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0099253969999999, 2024-09-21T23:16:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0184446170000001, 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27815 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761091+0000", 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761091+0000", 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761092+0000", 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4799999999999999e-07 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761094+0000", 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.452e-06 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761103+0000", 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6689999999999995e-06 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761113+0000", 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0540000000000001e-05 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761859+0000", 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000745454 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761862+0000", 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9869999999999999e-06 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0088561300000001, 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0174424339999999, 2024-09-21T23:16:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27864 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.771003+0000", 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.341e-05, 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0086650930000000004, 2024-09-21T23:16:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27864 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771003+0000", 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771003+0000", 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771004+0000", 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771005+0000", 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.607e-06 2024-09-21T23:16:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771010+0000", 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771013+0000", 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1219999999999999e-06 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26637 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752324+0000", 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0186926409999999, 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0274016070000001, 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26637 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752324+0000", 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752324+0000", 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752324+0000", 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752327+0000", 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.227e-06 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752333+0000", 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2920000000000001e-06 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752338+0000", 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3159999999999996e-06 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756443+0000", 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041046889999999999 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756449+0000", 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.801e-06 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26668 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754499+0000", 2024-09-21T23:16:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0165173000000001, 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.02529392, 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26668 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754499+0000", 2024-09-21T23:16:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754499+0000", 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754500+0000", 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754518+0000", 2024-09-21T23:16:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8389e-05 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754531+0000", 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2741e-05 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754537+0000", 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3960000000000003e-06 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757883+0000", 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033451039999999998 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757890+0000", 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4839999999999998e-06 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26697 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757750+0000", 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0132669919999999, 2024-09-21T23:16:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0221143500000001, 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26697 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757750+0000", 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757750+0000", 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757750+0000", 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.34e-07 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757753+0000", 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0730000000000001e-06 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757761+0000", 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6299999999999998e-06 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757766+0000", 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.293e-06 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759430+0000", 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00166428 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759434+0000", 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2619999999999999e-06 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26736 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759001+0000", 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0120153409999999, 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0209293269999999, 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26736 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759001+0000", 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759001+0000", 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759002+0000", 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759005+0000", 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1049999999999999e-06 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759013+0000", 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4370000000000007e-06 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759019+0000", 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760469+0000", 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014498099999999999 2024-09-21T23:16:18.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760472+0000", 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.123e-06 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27784 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760073+0000", 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0109440919999999, 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0199200879999999, 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27784 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760073+0000", 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760073+0000", 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760073+0000", 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0999999999999999e-07 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760075+0000", 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1770000000000001e-06 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760082+0000", 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8759999999999999e-06 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760088+0000", 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3080000000000004e-06 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761192+0000", 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001103396 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761195+0000", 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8609999999999998e-06 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27816 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761122+0000", 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0098944169999999, 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0189339019999999, 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27816 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761122+0000", 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761122+0000", 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761123+0000", 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1199999999999998e-07 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761125+0000", 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2919999999999998e-06 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761132+0000", 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2389999999999996e-06 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761137+0000", 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.181e-06 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761883+0000", 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074580299999999996 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761889+0000", 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5199999999999997e-06 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.008816055, 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0179245189999999, 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:18.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27865 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.771017+0000", 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967296, 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091894769999999997, 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27865 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771017+0000", 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771017+0000", 2024-09-21T23:16:18.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771017+0000", 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771019+0000", 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6139999999999999e-06 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771022+0000", 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.264e-06 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771025+0000", 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0529999999999998e-06 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26638 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752346+0000", 2024-09-21T23:16:18.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0186711129999999, 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0279217220000001, 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26638 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752346+0000", 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752346+0000", 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752346+0000", 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752348+0000", 2024-09-21T23:16:18.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.402e-06 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752356+0000", 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.447e-06 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752361+0000", 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756474+0000", 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004112821 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756480+0000", 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1330000000000004e-06 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26669 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754715+0000", 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0163013970000001, 2024-09-21T23:16:18.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0256233050000001, 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26669 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754715+0000", 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754715+0000", 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754716+0000", 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754720+0000", 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.968e-06 2024-09-21T23:16:18.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754739+0000", 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8593999999999998e-05 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754746+0000", 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9999999999999999e-06 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757935+0000", 2024-09-21T23:16:18.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031891350000000001 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757941+0000", 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6510000000000004e-06 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26698 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757789+0000", 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0132280010000001, 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0226200089999999, 2024-09-21T23:16:18.304 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26698 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757789+0000", 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757789+0000", 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757789+0000", 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757791+0000", 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757803+0000", 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1874e-05 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757809+0000", 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.575e-06 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759447+0000", 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00163861 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759450+0000", 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.79e-06 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26738 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759038+0000", 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0119783499999999, 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0214405959999999, 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26738 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759038+0000", 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759038+0000", 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759039+0000", 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.13e-07 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759042+0000", 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1939999999999998e-06 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759052+0000", 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0019e-05 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759057+0000", 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7960000000000003e-06 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760486+0000", 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014295880000000001 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760489+0000", 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9730000000000002e-06 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27785 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760111+0000", 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0109059600000001, 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0204404869999999, 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27785 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760111+0000", 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760111+0000", 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760111+0000", 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9800000000000004e-07 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760114+0000", 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0529999999999998e-06 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760122+0000", 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760128+0000", 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5149999999999997e-06 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761208+0000", 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001080073 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761211+0000", 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9000000000000002e-06 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27817 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761147+0000", 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0098696069999999, 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0194761880000001, 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27817 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.316 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761147+0000", 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761147+0000", 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761148+0000", 2024-09-21T23:16:18.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761150+0000", 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3989999999999998e-06 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761158+0000", 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9449999999999993e-06 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761163+0000", 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761921+0000", 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075793099999999999 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761927+0000", 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5989999999999998e-06 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0087915110000001, 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0184733159999999, 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:18.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:18.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27866 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.771029+0000", 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999881, 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097415069999999999, 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27866 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771029+0000", 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771029+0000", 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771029+0000", 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1e-07 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771031+0000", 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5600000000000001e-06 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771034+0000", 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.23e-06 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771037+0000", 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.326 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26639 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752730+0000", 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018286405, 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0281060710000001, 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26639 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752730+0000", 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752730+0000", 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752731+0000", 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.68e-07 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752734+0000", 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1200000000000002e-06 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752745+0000", 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0812000000000001e-05 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752752+0000", 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7410000000000003e-06 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756505+0000", 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003753284 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756510+0000", 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4419999999999997e-06 2024-09-21T23:16:18.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26670 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754776+0000", 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0162410129999999, 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026135408, 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26670 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754776+0000", 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754776+0000", 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754776+0000", 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0900000000000001e-07 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754780+0000", 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6109999999999998e-06 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754790+0000", 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6970000000000005e-06 2024-09-21T23:16:18.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754806+0000", 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6504e-05 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757977+0000", 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00317059 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757983+0000", 2024-09-21T23:16:18.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5760000000000002e-06 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26699 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757818+0000", 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013198855, 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023181087, 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26699 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757818+0000", 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757818+0000", 2024-09-21T23:16:18.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757818+0000", 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8700000000000001e-07 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757821+0000", 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6309999999999999e-06 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757829+0000", 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7589999999999997e-06 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757834+0000", 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5509999999999999e-06 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759473+0000", 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001638605 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759479+0000", 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9209999999999997e-06 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26740 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759064+0000", 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0119530290000001, 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0220099920000001, 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26740 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759064+0000", 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759064+0000", 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759064+0000", 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3799999999999999e-07 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759066+0000", 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7460000000000001e-06 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759070+0000", 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9870000000000001e-06 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759073+0000", 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9280000000000002e-06 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760503+0000", 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001429959 2024-09-21T23:16:18.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760505+0000", 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8540000000000001e-06 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27786 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760137+0000", 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0108797970000001, 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0210109060000001, 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.343 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27786 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760137+0000", 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760137+0000", 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760137+0000", 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760140+0000", 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4660000000000002e-06 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760147+0000", 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9929999999999998e-06 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760152+0000", 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2320000000000001e-06 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761223+0000", 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010712740000000001 2024-09-21T23:16:18.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761226+0000", 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8729999999999999e-06 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27818 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761178+0000", 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0098387120000001, 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0200373700000001, 2024-09-21T23:16:18.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27818 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761178+0000", 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761178+0000", 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761178+0000", 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.08e-07 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761181+0000", 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3989999999999998e-06 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761188+0000", 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2749999999999998e-06 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761193+0000", 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761974+0000", 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078103999999999999 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.657e-06 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0087537259999999, 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190209240000001, 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27867 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.771040+0000", 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999766, 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010313266, 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27867 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771040+0000", 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771040+0000", 2024-09-21T23:16:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771040+0000", 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0599999999999999e-07 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771042+0000", 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.567e-06 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771046+0000", 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3209999999999998e-06 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771049+0000", 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8279999999999999e-06 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26640 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752762+0000", 2024-09-21T23:16:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018254907, 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0286470430000001, 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26640 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752762+0000", 2024-09-21T23:16:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752762+0000", 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752762+0000", 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4799999999999999e-07 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752765+0000", 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5050000000000002e-06 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752772+0000", 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752778+0000", 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6609999999999997e-06 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756536+0000", 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037583310000000002 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756542+0000", 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26671 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754818+0000", 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0161987240000001, 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0266579899999999, 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26671 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754818+0000", 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754818+0000", 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754819+0000", 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.44e-07 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754822+0000", 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1659999999999998e-06 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754830+0000", 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9680000000000006e-06 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754844+0000", 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4545000000000001e-05 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758014+0000", 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031699039999999999 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758021+0000", 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5130000000000002e-06 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26700 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757873+0000", 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.01314342, 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0236646819999999, 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26700 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757873+0000", 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757873+0000", 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757874+0000", 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3000000000000001e-07 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757877+0000", 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0510000000000001e-06 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757886+0000", 2024-09-21T23:16:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2960000000000004e-06 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757893+0000", 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2509999999999997e-06 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759504+0000", 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016102499999999999 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759511+0000", 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5120000000000002e-06 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26742 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759076+0000", 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0119403360000001, 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0225295489999999, 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26742 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759076+0000", 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759076+0000", 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759077+0000", 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3900000000000001e-07 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759078+0000", 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.437e-06 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759082+0000", 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4920000000000002e-06 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759086+0000", 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.233e-06 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760518+0000", 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014321659999999999 2024-09-21T23:16:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760521+0000", 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7070000000000001e-06 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27787 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760170+0000", 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0108471640000001, 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0215073669999999, 2024-09-21T23:16:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27787 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760170+0000", 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760170+0000", 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760170+0000", 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760179+0000", 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2820000000000002e-06 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760188+0000", 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4200000000000007e-06 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760194+0000", 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8340000000000004e-06 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761239+0000", 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010459040000000001 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761242+0000", 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7199999999999998e-06 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27819 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761209+0000", 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009807304, 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0205302140000001, 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27819 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761209+0000", 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761209+0000", 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761210+0000", 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761213+0000", 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6649999999999999e-06 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761220+0000", 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7959999999999996e-06 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761226+0000", 2024-09-21T23:16:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3090000000000003e-06 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762017+0000", 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079101499999999995 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762023+0000", 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3810000000000004e-06 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27851 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762303+0000", 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.00871373, 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019500158, 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27851 2024-09-21T23:16:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762303+0000", 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762303+0000", 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762303+0000", 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.01e-07 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762306+0000", 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7099999999999999e-06 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762313+0000", 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762319+0000", 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0529999999999997e-06 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763176+0000", 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085756199999999997 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763183+0000", 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4479999999999999e-06 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27868 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.771053+0000", 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999638, 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010820184, 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27868 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771053+0000", 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771053+0000", 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771053+0000", 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1899999999999999e-07 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771055+0000", 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4750000000000001e-06 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771058+0000", 2024-09-21T23:16:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2540000000000001e-06 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771061+0000", 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.18e-06 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26641 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752786+0000", 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018230427, 2024-09-21T23:16:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0291422269999999, 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26641 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752786+0000", 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752786+0000", 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752787+0000", 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.68e-07 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752789+0000", 2024-09-21T23:16:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4530000000000001e-06 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752797+0000", 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3939999999999998e-06 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752802+0000", 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2310000000000004e-06 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756567+0000", 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037653610000000001 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756573+0000", 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8270000000000003e-06 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26672 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.754856+0000", 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.016161122, 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.027137757, 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26672 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754856+0000", 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754856+0000", 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754856+0000", 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2600000000000002e-07 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754859+0000", 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.196e-06 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754867+0000", 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9030000000000004e-06 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.754882+0000", 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5136e-05 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758055+0000", 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031724650000000002 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758062+0000", 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8229999999999997e-06 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26701 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757931+0000", 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0130859969999999, 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0241274069999999, 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26701 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757931+0000", 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757931+0000", 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757931+0000", 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5199999999999997e-07 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757934+0000", 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1080000000000001e-06 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757944+0000", 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9590000000000007e-06 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757951+0000", 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3389999999999998e-06 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759543+0000", 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015918880000000001 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759553+0000", 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0740000000000001e-05 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26744 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759090+0000", 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0119267110000001, 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023036968, 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26744 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759090+0000", 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759090+0000", 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759090+0000", 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-21T23:16:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759092+0000", 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.443e-06 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759095+0000", 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5109999999999999e-06 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759099+0000", 2024-09-21T23:16:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.411e-06 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760535+0000", 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014359170000000001 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760537+0000", 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8890000000000002e-06 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27788 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760204+0000", 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010812912, 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0219869260000001, 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27788 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760204+0000", 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760204+0000", 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760204+0000", 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000001e-07 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760207+0000", 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7640000000000001e-06 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760216+0000", 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1139999999999995e-06 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760223+0000", 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7229999999999998e-06 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761254+0000", 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010315910000000001 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761257+0000", 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8109999999999999e-06 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27820 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761241+0000", 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0097761439999999, 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021010864, 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27820 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761241+0000", 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761241+0000", 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761241+0000", 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761243+0000", 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4770000000000002e-06 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761251+0000", 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4939999999999997e-06 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761256+0000", 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4879999999999998e-06 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762052+0000", 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000795852 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762058+0000", 2024-09-21T23:16:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1659999999999999e-06 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27852 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762339+0000", 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0086780150000001, 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019973953, 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27852 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762339+0000", 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762339+0000", 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762339+0000", 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762342+0000", 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7329999999999999e-06 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762350+0000", 2024-09-21T23:16:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6839999999999996e-06 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762355+0000", 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4439999999999999e-06 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763207+0000", 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085230699999999996 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763213+0000", 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9900000000000002e-06 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27869 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.771065+0000", 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999523, 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011315254, 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27869 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771065+0000", 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.771065+0000", 2024-09-21T23:16:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.774861+0000", 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037965350000000002 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.774867+0000", 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3759999999999996e-06 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.774882+0000", 2024-09-21T23:16:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4113e-05 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.774888+0000", 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8140000000000003e-06 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26642 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752810+0000", 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0182070679999999, 2024-09-21T23:16:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0296258389999999, 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26642 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752810+0000", 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752810+0000", 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752810+0000", 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3599999999999999e-07 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752812+0000", 2024-09-21T23:16:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2350000000000002e-06 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752819+0000", 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1520000000000003e-06 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752825+0000", 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3229999999999997e-06 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756599+0000", 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037740410000000001 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756610+0000", 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0848e-05 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26673 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.755012+0000", 2024-09-21T23:16:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0160049600000001, 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0274896170000001, 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26673 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755012+0000", 2024-09-21T23:16:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755012+0000", 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755013+0000", 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0500000000000002e-07 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755017+0000", 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7539999999999997e-06 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755027+0000", 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8560000000000002e-06 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.755035+0000", 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9570000000000002e-06 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758080+0000", 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030444650000000001 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758083+0000", 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.095e-06 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26746 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759103+0000", 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0119140369999999, 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0234720530000001, 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26746 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759103+0000", 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759103+0000", 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759103+0000", 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759104+0000", 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3349999999999999e-06 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759108+0000", 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.326e-06 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759110+0000", 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.836e-06 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760553+0000", 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014422300000000001 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760556+0000", 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27789 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760232+0000", 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010784554, 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0224056530000001, 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27789 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760232+0000", 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760232+0000", 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760233+0000", 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760235+0000", 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3029999999999998e-06 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760242+0000", 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1350000000000003e-06 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760247+0000", 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2560000000000002e-06 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761271+0000", 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010233569999999999 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761274+0000", 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8669999999999998e-06 2024-09-21T23:16:18.441 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27821 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761273+0000", 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009744135, 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0214407649999999, 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27821 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761273+0000", 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761273+0000", 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761273+0000", 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2100000000000002e-07 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761276+0000", 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5579999999999999e-06 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761283+0000", 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5780000000000001e-06 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761289+0000", 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1879999999999999e-06 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762091+0000", 2024-09-21T23:16:18.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080184600000000005 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762097+0000", 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1099999999999999e-06 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27853 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762374+0000", 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.00864245, 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0204043350000001, 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27853 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762374+0000", 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762374+0000", 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762375+0000", 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762377+0000", 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.587e-06 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762386+0000", 2024-09-21T23:16:18.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3180000000000007e-06 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762391+0000", 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6250000000000004e-06 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763242+0000", 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085103299999999995 2024-09-21T23:16:18.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763249+0000", 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4370000000000004e-06 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27870 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.774904+0000", 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9961128, 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0079402540000000008, 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27870 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.774904+0000", 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.774904+0000", 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778699+0000", 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037954199999999999 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778704+0000", 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.758e-06 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778716+0000", 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1770000000000001e-05 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778722+0000", 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.156e-06 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26643 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752833+0000", 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018183947, 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300643279999999, 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.455 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26643 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752833+0000", 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752833+0000", 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752833+0000", 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.51e-07 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752836+0000", 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7070000000000001e-06 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752844+0000", 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8639999999999996e-06 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752851+0000", 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4680000000000003e-06 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756638+0000", 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037871599999999999 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756644+0000", 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9479999999999996e-06 2024-09-21T23:16:18.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26702 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757974+0000", 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013042912, 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249919750000001, 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.460 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26702 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757974+0000", 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757974+0000", 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757974+0000", 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8800000000000003e-07 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757977+0000", 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6019999999999998e-06 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757985+0000", 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.191e-06 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757991+0000", 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.981e-06 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759588+0000", 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015967570000000001 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759595+0000", 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.255e-06 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.464 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26748 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759114+0000", 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0119028029999999, 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023919652, 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.465 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26748 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759114+0000", 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759114+0000", 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759114+0000", 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759116+0000", 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4249999999999999e-06 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759121+0000", 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759126+0000", 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9470000000000002e-06 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760568+0000", 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001441702 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760571+0000", 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8710000000000001e-06 2024-09-21T23:16:18.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27790 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760269+0000", 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0107480010000001, 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0228298440000001, 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27790 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760269+0000", 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760269+0000", 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760269+0000", 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2099999999999997e-07 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760272+0000", 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.016e-06 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760280+0000", 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.174e-06 2024-09-21T23:16:18.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760287+0000", 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6250000000000001e-06 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761286+0000", 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099851999999999992 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761290+0000", 2024-09-21T23:16:18.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9979999999999997e-06 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27822 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761305+0000", 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009711507, 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0218588580000001, 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27822 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761305+0000", 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761305+0000", 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761306+0000", 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-07 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761309+0000", 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761316+0000", 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.554e-06 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761322+0000", 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8019999999999997e-06 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762130+0000", 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000808233 2024-09-21T23:16:18.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762136+0000", 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27854 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762411+0000", 2024-09-21T23:16:18.477 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0086053559999999, 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0208180520000001, 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27854 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762411+0000", 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762411+0000", 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762412+0000", 2024-09-21T23:16:18.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999999e-07 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762415+0000", 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.756e-06 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762422+0000", 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4510000000000003e-06 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762427+0000", 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1760000000000001e-06 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763268+0000", 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084118099999999998 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763272+0000", 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2600000000000001e-06 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27871 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778750+0000", 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9922662, 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045446729999999999, 2024-09-21T23:16:18.482 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27871 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778750+0000", 2024-09-21T23:16:18.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778750+0000", 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778751+0000", 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-21T23:16:18.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778754+0000", 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8310000000000001e-06 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778760+0000", 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778764+0000", 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2250000000000002e-06 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26644 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752873+0000", 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0181434869999999, 2024-09-21T23:16:18.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0304790859999999, 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26644 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752873+0000", 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752873+0000", 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752874+0000", 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.09e-07 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752897+0000", 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2819999999999998e-05 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752908+0000", 2024-09-21T23:16:18.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1404e-05 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752917+0000", 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756670+0000", 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037531629999999999 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756676+0000", 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26703 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758010+0000", 2024-09-21T23:16:18.491 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0130067570000001, 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025407497, 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26703 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758010+0000", 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758010+0000", 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758010+0000", 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9399999999999995e-07 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758013+0000", 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7949999999999999e-06 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758022+0000", 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758029+0000", 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0389999999999999e-06 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759628+0000", 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015983010000000001 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759646+0000", 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8714000000000001e-05 2024-09-21T23:16:18.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26751 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759135+0000", 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011882129, 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.024349521, 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26751 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759135+0000", 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759135+0000", 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759135+0000", 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6800000000000001e-07 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759137+0000", 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.508e-06 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759146+0000", 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5050000000000007e-06 2024-09-21T23:16:18.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759151+0000", 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0930000000000002e-06 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760585+0000", 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014337670000000001 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760588+0000", 2024-09-21T23:16:18.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8880000000000001e-06 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27791 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760305+0000", 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.01071209, 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0232441619999999, 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27791 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760305+0000", 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760305+0000", 2024-09-21T23:16:18.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760305+0000", 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760308+0000", 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760316+0000", 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7069999999999992e-06 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760321+0000", 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1309999999999997e-06 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761303+0000", 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098230499999999994 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761306+0000", 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.757e-06 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27823 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761331+0000", 2024-09-21T23:16:18.504 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009685274, 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.022283718, 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27823 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761331+0000", 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761331+0000", 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761332+0000", 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-21T23:16:18.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761342+0000", 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7380000000000006e-06 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761349+0000", 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2099999999999996e-06 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761354+0000", 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4859999999999997e-06 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762164+0000", 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080971000000000005 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762170+0000", 2024-09-21T23:16:18.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8499999999999999e-06 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27855 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762445+0000", 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.00857167, 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021237349, 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.509 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27855 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762445+0000", 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762445+0000", 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762445+0000", 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762449+0000", 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.089e-06 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762456+0000", 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8350000000000004e-06 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762462+0000", 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4859999999999997e-06 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763283+0000", 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082159799999999999 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763286+0000", 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27872 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778774+0000", 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9922423, 2024-09-21T23:16:18.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049770190000000001, 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27872 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778774+0000", 2024-09-21T23:16:18.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778774+0000", 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778775+0000", 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8599999999999999e-07 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778776+0000", 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9180000000000001e-06 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778783+0000", 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1820000000000003e-06 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778786+0000", 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.631e-06 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26645 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752930+0000", 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.018086552, 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0308787639999999, 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26645 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752930+0000", 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752930+0000", 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752931+0000", 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752934+0000", 2024-09-21T23:16:18.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3340000000000002e-06 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752942+0000", 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1699999999999997e-06 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752948+0000", 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.981e-06 2024-09-21T23:16:18.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756701+0000", 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003753134 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756708+0000", 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1109999999999996e-06 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.522 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26674 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.756889+0000", 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.014128076, 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0269878690000001, 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26674 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756889+0000", 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756889+0000", 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756889+0000", 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8899999999999998e-07 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756894+0000", 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7829999999999998e-06 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756905+0000", 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0413e-05 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756912+0000", 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3490000000000003e-06 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758105+0000", 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011928679999999999 2024-09-21T23:16:18.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758111+0000", 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9490000000000001e-06 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26704 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758056+0000", 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0129603469999999, 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025888723, 2024-09-21T23:16:18.527 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26704 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758056+0000", 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758056+0000", 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758057+0000", 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8800000000000002e-07 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758067+0000", 2024-09-21T23:16:18.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7750000000000004e-06 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758077+0000", 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0159999999999999e-05 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758083+0000", 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4139999999999999e-06 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759682+0000", 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015987849999999999 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759690+0000", 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9270000000000005e-06 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26753 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759159+0000", 2024-09-21T23:16:18.531 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0118574769999999, 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.024854457, 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26753 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759159+0000", 2024-09-21T23:16:18.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759159+0000", 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759160+0000", 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759162+0000", 2024-09-21T23:16:18.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5399999999999998e-06 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759169+0000", 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5529999999999998e-06 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759174+0000", 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8369999999999996e-06 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760601+0000", 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001427397 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760604+0000", 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8229999999999999e-06 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27792 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.535 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760336+0000", 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010680319, 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0237448330000001, 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27792 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.536 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760336+0000", 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760336+0000", 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760337+0000", 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760340+0000", 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.689e-06 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760347+0000", 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3560000000000004e-06 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760352+0000", 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761318+0000", 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096658499999999995 2024-09-21T23:16:18.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761321+0000", 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8439999999999998e-06 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27824 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761376+0000", 2024-09-21T23:16:18.540 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009640962, 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.022773028, 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27824 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761376+0000", 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761376+0000", 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761376+0000", 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761379+0000", 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761387+0000", 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0350000000000001e-06 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761393+0000", 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7649999999999999e-06 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762196+0000", 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080323199999999997 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762202+0000", 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1009999999999997e-06 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27856 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762478+0000", 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.008538538, 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.021739669, 2024-09-21T23:16:18.545 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27856 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762478+0000", 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762478+0000", 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762479+0000", 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3000000000000001e-07 2024-09-21T23:16:18.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762481+0000", 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4320000000000002e-06 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762488+0000", 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0269999999999998e-06 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762493+0000", 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1449999999999999e-06 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763299+0000", 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080580199999999999 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763302+0000", 2024-09-21T23:16:18.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9639999999999999e-06 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27873 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778791+0000", 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9922252, 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054914289999999999, 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27873 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778791+0000", 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778791+0000", 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778792+0000", 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6600000000000003e-07 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778793+0000", 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.454e-06 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778797+0000", 2024-09-21T23:16:18.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2030000000000002e-06 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778801+0000", 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4209999999999999e-06 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26646 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752960+0000", 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0180570120000001, 2024-09-21T23:16:18.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.031379957, 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26646 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752960+0000", 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752960+0000", 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752960+0000", 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0900000000000002e-07 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752963+0000", 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1640000000000001e-06 2024-09-21T23:16:18.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752979+0000", 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5600999999999999e-05 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752986+0000", 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7630000000000003e-06 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756735+0000", 2024-09-21T23:16:18.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037488479999999999 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756741+0000", 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.139e-06 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26675 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.756938+0000", 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0140784839999999, 2024-09-21T23:16:18.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.027469446, 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26675 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756938+0000", 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756938+0000", 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756939+0000", 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756943+0000", 2024-09-21T23:16:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5269999999999999e-06 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756951+0000", 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5459999999999992e-06 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756958+0000", 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6069999999999996e-06 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758137+0000", 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001179842 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758144+0000", 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26705 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758136+0000", 2024-09-21T23:16:18.561 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012880878, 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026337343, 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26705 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758136+0000", 2024-09-21T23:16:18.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758136+0000", 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758136+0000", 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8999999999999996e-07 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758140+0000", 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5889999999999999e-06 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758149+0000", 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3829999999999998e-06 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758156+0000", 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7519999999999999e-06 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759721+0000", 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015653080000000001 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759725+0000", 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9009999999999996e-06 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26755 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759185+0000", 2024-09-21T23:16:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0118317429999999, 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025357597, 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26755 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759185+0000", 2024-09-21T23:16:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759185+0000", 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759185+0000", 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759188+0000", 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3949999999999999e-06 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759197+0000", 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3680000000000008e-06 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759203+0000", 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.75e-06 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760617+0000", 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001413779 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760620+0000", 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8990000000000001e-06 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27793 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760370+0000", 2024-09-21T23:16:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010646339, 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.024239034, 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27793 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760370+0000", 2024-09-21T23:16:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760370+0000", 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760371+0000", 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5200000000000002e-07 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760374+0000", 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8040000000000002e-06 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760382+0000", 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.388e-06 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760388+0000", 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761333+0000", 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094546699999999997 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761336+0000", 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27825 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761413+0000", 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.009604119, 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0232658130000001, 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27825 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761413+0000", 2024-09-21T23:16:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761413+0000", 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761413+0000", 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999998e-07 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761416+0000", 2024-09-21T23:16:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.672e-06 2024-09-21T23:16:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761424+0000", 2024-09-21T23:16:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9799999999999998e-06 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761430+0000", 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9519999999999999e-06 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762230+0000", 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080035100000000004 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762236+0000", 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1120000000000001e-06 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27857 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762514+0000", 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0085026159999999, 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0222326209999999, 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27857 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762514+0000", 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762514+0000", 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762515+0000", 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7819999999999998e-06 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762526+0000", 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0380000000000005e-06 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762532+0000", 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5609999999999998e-06 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763324+0000", 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079242099999999997 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763328+0000", 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.18e-06 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27874 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778805+0000", 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9922113, 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0060110809999999997, 2024-09-21T23:16:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27874 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778805+0000", 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778805+0000", 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778805+0000", 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5699999999999999e-07 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778807+0000", 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.44e-06 2024-09-21T23:16:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778811+0000", 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778815+0000", 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6820000000000001e-06 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26647 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.752996+0000", 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0180206329999999, 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0318787659999999, 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26647 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752996+0000", 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752996+0000", 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.752997+0000", 2024-09-21T23:16:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9900000000000001e-07 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753000+0000", 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0939999999999999e-06 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753008+0000", 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8159999999999994e-06 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753014+0000", 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0759999999999999e-06 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756771+0000", 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037569190000000001 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756777+0000", 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3929999999999996e-06 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26676 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.756992+0000", 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.014024695, 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0279543680000001, 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26676 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756992+0000", 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756992+0000", 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756993+0000", 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0399999999999996e-07 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756996+0000", 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4520000000000002e-06 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757005+0000", 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1079999999999999e-06 2024-09-21T23:16:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757010+0000", 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8720000000000001e-06 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758171+0000", 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001160838 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758177+0000", 2024-09-21T23:16:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8109999999999999e-06 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26706 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758168+0000", 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012848787, 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026846554, 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26706 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758168+0000", 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758168+0000", 2024-09-21T23:16:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758169+0000", 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7400000000000003e-07 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758171+0000", 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7429999999999998e-06 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758179+0000", 2024-09-21T23:16:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5850000000000002e-06 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758185+0000", 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9070000000000004e-06 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759740+0000", 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015551549999999999 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759743+0000", 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.836e-06 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27762 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759212+0000", 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0118043000000001, 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0258692540000001, 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27762 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759212+0000", 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759212+0000", 2024-09-21T23:16:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759213+0000", 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9000000000000002e-07 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759215+0000", 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.452e-06 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759224+0000", 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5350000000000004e-06 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759229+0000", 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760632+0000", 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014024560000000001 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760635+0000", 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8480000000000001e-06 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27794 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760404+0000", 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010612375, 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.024746047, 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27794 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760404+0000", 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760404+0000", 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760405+0000", 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760408+0000", 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.903e-06 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760415+0000", 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6079999999999998e-06 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760421+0000", 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.446e-06 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761349+0000", 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092855100000000001 2024-09-21T23:16:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761355+0000", 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7089999999999999e-06 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27826 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761448+0000", 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0095684009999999, 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023769787, 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27826 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761448+0000", 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761448+0000", 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761449+0000", 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3599999999999999e-07 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761451+0000", 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6469999999999998e-06 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761459+0000", 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4560000000000003e-06 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761464+0000", 2024-09-21T23:16:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762270+0000", 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080580300000000001 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762276+0000", 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27858 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762550+0000", 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0084671789999999, 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0227282150000001, 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27858 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762550+0000", 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762550+0000", 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762550+0000", 2024-09-21T23:16:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6100000000000002e-07 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762552+0000", 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4269999999999998e-06 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762559+0000", 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762564+0000", 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9869999999999999e-06 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763340+0000", 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077609599999999999 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763344+0000", 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1889999999999999e-06 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27875 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778819+0000", 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.992198, 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006520456, 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27875 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778819+0000", 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778819+0000", 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778819+0000", 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.53e-07 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778824+0000", 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4259999999999996e-06 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778828+0000", 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0470000000000004e-06 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778831+0000", 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2890000000000002e-06 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26648 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.753024+0000", 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.017992891, 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0323742549999999, 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26648 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753024+0000", 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753024+0000", 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753024+0000", 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7100000000000002e-07 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753027+0000", 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7329999999999999e-06 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753034+0000", 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1189999999999999e-06 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.753040+0000", 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756803+0000", 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037636919999999999 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.756809+0000", 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9719999999999997e-06 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.785384+0000", 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0285744800000001 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26677 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757034+0000", 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0139831500000001, 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0284444930000001, 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26677 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757034+0000", 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757034+0000", 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757034+0000", 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757037+0000", 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2449999999999998e-06 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757046+0000", 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5089999999999993e-06 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757052+0000", 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5259999999999999e-06 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758205+0000", 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001152672 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758211+0000", 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7039999999999999e-06 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.627 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26707 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758199+0000", 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0128177169999999, 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0273462120000001, 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26707 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758199+0000", 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758199+0000", 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758199+0000", 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8500000000000002e-07 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758203+0000", 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758212+0000", 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758218+0000", 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0669999999999997e-06 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759757+0000", 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001539059 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759760+0000", 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8200000000000001e-06 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27763 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759238+0000", 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0117785850000001, 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0263744669999999, 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27763 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759238+0000", 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759238+0000", 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759238+0000", 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4299999999999999e-07 2024-09-21T23:16:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759249+0000", 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0954999999999999e-05 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759261+0000", 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2031999999999999e-05 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759269+0000", 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3649999999999998e-06 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760651+0000", 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001382037 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760654+0000", 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9660000000000001e-06 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27795 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760440+0000", 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0105764260000001, 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025246122, 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27795 2024-09-21T23:16:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760440+0000", 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760440+0000", 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760441+0000", 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760444+0000", 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3170000000000002e-06 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760458+0000", 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3991999999999999e-05 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760465+0000", 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.72e-06 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761379+0000", 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091400499999999996 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761385+0000", 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8980000000000001e-06 2024-09-21T23:16:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27827 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761484+0000", 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0095329200000001, 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0242703930000001, 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27827 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761484+0000", 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761484+0000", 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761484+0000", 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761487+0000", 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761495+0000", 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1389999999999995e-06 2024-09-21T23:16:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761501+0000", 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9850000000000003e-06 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762308+0000", 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080701600000000003 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762314+0000", 2024-09-21T23:16:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0839999999999997e-06 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27859 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762587+0000", 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.008429631, 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023232508, 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27859 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762587+0000", 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762587+0000", 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762588+0000", 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4799999999999999e-07 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762596+0000", 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0069999999999997e-06 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762604+0000", 2024-09-21T23:16:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0930000000000003e-06 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762610+0000", 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.874e-06 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763371+0000", 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076139299999999997 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763377+0000", 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0939999999999996e-06 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27876 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778835+0000", 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9921818, 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070477339999999999, 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27876 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778835+0000", 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778835+0000", 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778835+0000", 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.53e-07 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778837+0000", 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.446e-06 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778841+0000", 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0260000000000001e-06 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778844+0000", 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3850000000000002e-06 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26678 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757062+0000", 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013954287, 2024-09-21T23:16:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0288573620000001, 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26678 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757062+0000", 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757062+0000", 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757063+0000", 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1899999999999998e-07 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757065+0000", 2024-09-21T23:16:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3329999999999999e-06 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757072+0000", 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1860000000000003e-06 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757078+0000", 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6470000000000003e-06 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758240+0000", 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001161762 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758246+0000", 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2009999999999996e-06 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26708 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758236+0000", 2024-09-21T23:16:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0127804899999999, 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0277189360000001, 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26708 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758236+0000", 2024-09-21T23:16:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758236+0000", 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758237+0000", 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3799999999999998e-07 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758239+0000", 2024-09-21T23:16:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2120000000000002e-06 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758244+0000", 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4260000000000002e-06 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758247+0000", 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2710000000000001e-06 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759775+0000", 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015278539999999999 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759781+0000", 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27764 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759281+0000", 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0117360980000001, 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.026709383, 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27764 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759281+0000", 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759281+0000", 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759281+0000", 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759284+0000", 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.836e-06 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759312+0000", 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7707000000000001e-05 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759322+0000", 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0237999999999999e-05 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760668+0000", 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001346135 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760671+0000", 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9170000000000002e-06 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.663 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27796 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760476+0000", 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0105403159999999, 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025548847, 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27796 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760476+0000", 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760476+0000", 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760477+0000", 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2399999999999999e-07 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760479+0000", 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6029999999999999e-06 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760487+0000", 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6480000000000003e-06 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760493+0000", 2024-09-21T23:16:18.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6450000000000002e-06 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761416+0000", 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00092309599999999999 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761422+0000", 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9599999999999997e-06 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27828 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761519+0000", 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0094977970000001, 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.024569785, 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27828 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761519+0000", 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761519+0000", 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761519+0000", 2024-09-21T23:16:18.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4000000000000002e-07 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761522+0000", 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6709999999999999e-06 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761530+0000", 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8480000000000001e-06 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761536+0000", 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8760000000000002e-06 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762343+0000", 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080716499999999999 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762349+0000", 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0859999999999998e-06 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27860 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762618+0000", 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0083986570000001, 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.023536741, 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27860 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.672 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762618+0000", 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762618+0000", 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762618+0000", 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2599999999999998e-07 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762621+0000", 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2510000000000001e-06 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762627+0000", 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6379999999999998e-06 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762632+0000", 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0470000000000001e-06 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763397+0000", 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000764246 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763400+0000", 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0800000000000002e-06 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27877 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778850+0000", 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.992167, 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0073709910000000003, 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27877 2024-09-21T23:16:18.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778850+0000", 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778850+0000", 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778850+0000", 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.65e-07 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778851+0000", 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.438e-06 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778855+0000", 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0060000000000003e-06 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778859+0000", 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2959999999999999e-06 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26679 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757114+0000", 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0139025260000001, 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0291702869999999, 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26679 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757114+0000", 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757114+0000", 2024-09-21T23:16:18.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757115+0000", 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6100000000000001e-07 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757118+0000", 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6669999999999998e-06 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757126+0000", 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3520000000000001e-06 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757129+0000", 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7519999999999998e-06 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758277+0000", 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011472800000000001 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758297+0000", 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9868e-05 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26709 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758275+0000", 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.012741305, 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0280800210000001, 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26709 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758275+0000", 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758275+0000", 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758276+0000", 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758279+0000", 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758291+0000", 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1832e-05 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758300+0000", 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5250000000000005e-06 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759799+0000", 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014998120000000001 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759802+0000", 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0850000000000001e-06 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27765 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759337+0000", 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0116801449999999, 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.027087106, 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27765 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759337+0000", 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759337+0000", 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759337+0000", 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2900000000000003e-07 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759342+0000", 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.233e-06 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759350+0000", 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7560000000000005e-06 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759356+0000", 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8250000000000001e-06 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760684+0000", 2024-09-21T23:16:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001327614 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760687+0000", 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8789999999999999e-06 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27797 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760501+0000", 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010515941, 2024-09-21T23:16:18.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0259910080000001, 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27797 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760501+0000", 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760501+0000", 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760501+0000", 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.08e-07 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760504+0000", 2024-09-21T23:16:18.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3070000000000001e-06 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760511+0000", 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0169999999999999e-06 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760516+0000", 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5400000000000003e-06 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761453+0000", 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093717099999999999 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761459+0000", 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27829 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761554+0000", 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0094628210000001, 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0250061340000001, 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27829 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761554+0000", 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761554+0000", 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761554+0000", 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0499999999999999e-07 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761557+0000", 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5440000000000001e-06 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761565+0000", 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761571+0000", 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6550000000000001e-06 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762381+0000", 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080983699999999999 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762387+0000", 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2149999999999998e-06 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.699 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27861 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762656+0000", 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.008360908, 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0239720670000001, 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27861 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762656+0000", 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762656+0000", 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762656+0000", 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.34e-07 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762659+0000", 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7769999999999998e-06 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762668+0000", 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1009999999999998e-06 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762674+0000", 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763421+0000", 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074750700000000001 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763427+0000", 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5860000000000004e-06 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.703 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27878 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778863+0000", 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9921536, 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0079259180000000005, 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.704 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27878 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778863+0000", 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778863+0000", 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778863+0000", 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6300000000000001e-07 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778864+0000", 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.358e-06 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778868+0000", 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9929999999999997e-06 2024-09-21T23:16:18.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778872+0000", 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2839999999999998e-06 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26680 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757137+0000", 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0138801129999999, 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.029712148, 2024-09-21T23:16:18.707 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26680 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757137+0000", 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757137+0000", 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757137+0000", 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.53e-07 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757157+0000", 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0372000000000001e-05 2024-09-21T23:16:18.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757169+0000", 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2149e-05 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757176+0000", 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6270000000000002e-06 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758353+0000", 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011771469999999999 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758361+0000", 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26710 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758327+0000", 2024-09-21T23:16:18.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0126899490000001, 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0285899999999999, 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26710 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758327+0000", 2024-09-21T23:16:18.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758327+0000", 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758327+0000", 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5000000000000003e-07 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758331+0000", 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2799999999999999e-06 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758341+0000", 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0356999999999999e-05 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758347+0000", 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2099999999999998e-06 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759819+0000", 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001472232 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759822+0000", 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.892e-06 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27766 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759366+0000", 2024-09-21T23:16:18.715 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0116505280000001, 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0276071920000001, 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27766 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759366+0000", 2024-09-21T23:16:18.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759366+0000", 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759367+0000", 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8599999999999998e-07 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759370+0000", 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.892e-06 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759379+0000", 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3510000000000008e-06 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759385+0000", 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9989999999999997e-06 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760700+0000", 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001315091 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760703+0000", 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.926e-06 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.719 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27798 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760524+0000", 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.010492664, 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0264850830000001, 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27798 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760524+0000", 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760524+0000", 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760524+0000", 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6199999999999999e-07 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760527+0000", 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4530000000000001e-06 2024-09-21T23:16:18.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760534+0000", 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8519999999999998e-06 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760539+0000", 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3170000000000001e-06 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761488+0000", 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094867499999999995 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761493+0000", 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6280000000000002e-06 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27830 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761589+0000", 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0094273460000001, 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025456336, 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27830 2024-09-21T23:16:18.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761589+0000", 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761589+0000", 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761590+0000", 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761593+0000", 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.779e-06 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761601+0000", 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.909e-06 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761606+0000", 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762415+0000", 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080851600000000001 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762420+0000", 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7259999999999999e-06 2024-09-21T23:16:18.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27879 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778876+0000", 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9921408, 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0082053439999999998, 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.729 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27879 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778876+0000", 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778876+0000", 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778876+0000", 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.91e-07 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778878+0000", 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4389999999999999e-06 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778882+0000", 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9310000000000001e-06 2024-09-21T23:16:18.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778886+0000", 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8570000000000003e-06 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26681 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757186+0000", 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0138303689999999, 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0299262469999999, 2024-09-21T23:16:18.732 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26681 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757186+0000", 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757186+0000", 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757187+0000", 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757189+0000", 2024-09-21T23:16:18.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3949999999999999e-06 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757197+0000", 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1079999999999993e-06 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757203+0000", 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.524e-06 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758408+0000", 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012046909999999999 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758416+0000", 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5380000000000002e-06 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26711 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758357+0000", 2024-09-21T23:16:18.736 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0126596859999999, 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.028790192, 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26711 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758357+0000", 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758357+0000", 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758357+0000", 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1300000000000001e-07 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758360+0000", 2024-09-21T23:16:18.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4430000000000002e-06 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758368+0000", 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758373+0000", 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3600000000000004e-06 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759835+0000", 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001461487 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759838+0000", 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8870000000000001e-06 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27767 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759394+0000", 2024-09-21T23:16:18.740 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0116226420000001, 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0277877369999999, 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27767 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759394+0000", 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759394+0000", 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759394+0000", 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3000000000000002e-07 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759397+0000", 2024-09-21T23:16:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3439999999999999e-06 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759404+0000", 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0310000000000001e-06 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759411+0000", 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3749999999999997e-06 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760716+0000", 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013049030000000001 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760719+0000", 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.915e-06 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27799 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760554+0000", 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0104623749999999, 2024-09-21T23:16:18.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0266620179999999, 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27799 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760554+0000", 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760554+0000", 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760555+0000", 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2199999999999999e-07 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760566+0000", 2024-09-21T23:16:18.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.081e-05 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760574+0000", 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9920000000000007e-06 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760580+0000", 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.562e-06 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761523+0000", 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094290699999999999 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761529+0000", 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000001e-06 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:18.748 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.00939103, 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.025624884, 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:18.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27880 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778907+0000", 2024-09-21T23:16:18.752 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9921103, 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0083792769999999992, 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27880 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778907+0000", 2024-09-21T23:16:18.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778907+0000", 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778907+0000", 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.53e-07 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778909+0000", 2024-09-21T23:16:18.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9240000000000001e-06 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778916+0000", 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6969999999999996e-06 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778919+0000", 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6279999999999998e-06 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26682 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.757223+0000", 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013793934, 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300967569999999, 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26682 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.756 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757223+0000", 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757223+0000", 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757223+0000", 2024-09-21T23:16:18.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1800000000000002e-07 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757225+0000", 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0870000000000001e-06 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757231+0000", 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4260000000000002e-06 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.757234+0000", 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3909999999999998e-06 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758454+0000", 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012203680000000001 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758461+0000", 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.64e-06 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:26712 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.758398+0000", 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0126191680000001, 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.028956845, 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 26712 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758398+0000", 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758398+0000", 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758398+0000", 2024-09-21T23:16:18.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.75e-07 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758401+0000", 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9079999999999999e-06 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758409+0000", 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2579999999999995e-06 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.758415+0000", 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.003e-06 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759852+0000", 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001436295 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759854+0000", 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6429999999999999e-06 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27768 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.759511+0000", 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011505493, 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0278777910000001, 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27768 2024-09-21T23:16:18.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759511+0000", 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759511+0000", 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759512+0000", 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759516+0000", 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9489999999999998e-06 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759533+0000", 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7283999999999999e-05 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.759541+0000", 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1089999999999998e-06 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760738+0000", 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011968949999999999 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760744+0000", 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:18.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27800 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.760590+0000", 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0104266449999999, 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0268335660000001, 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27800 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760590+0000", 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760590+0000", 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760591+0000", 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9500000000000003e-07 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760593+0000", 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5110000000000002e-06 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760600+0000", 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.17e-06 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.760606+0000", 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4999999999999999e-06 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761559+0000", 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095282799999999996 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761565+0000", 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0000000000000002e-06 2024-09-21T23:16:18.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0093457729999999, 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.02578942, 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:18.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:18.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27881 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.778931+0000", 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9920855, 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.008564459, 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:18.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27881 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778931+0000", 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778931+0000", 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778931+0000", 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8299999999999998e-07 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778933+0000", 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7230000000000001e-06 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778941+0000", 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4839999999999998e-06 2024-09-21T23:16:18.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.778947+0000", 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2759999999999997e-06 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 241 2024-09-21T23:16:18.779 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:18.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:18.781 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:18.775+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:18.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:18.797 INFO:teuthology.orchestra.run.smithi170.stdout:2711 read 0x3b12000 thru 0x3b21598 (0xf599 bytes) 2024-09-21T23:16:18.849 INFO:teuthology.orchestra.run.smithi170.stdout:2714 write 0x9d9b400 thru 0x9da5247 (0x9e48 bytes) 2024-09-21T23:16:18.850 INFO:teuthology.orchestra.run.smithi170.stdout:2716 write 0x44d1c00 thru 0x44d1ce9 (0xea bytes) 2024-09-21T23:16:18.851 INFO:teuthology.orchestra.run.smithi170.stdout:2717 punch from 0x4d19a00 to 0x4d25600, (0xbc00 bytes) 2024-09-21T23:16:18.852 INFO:teuthology.orchestra.run.smithi170.stdout:2718 punch from 0x3e6be00 to 0x3e72400, (0x6600 bytes) 2024-09-21T23:16:18.852 INFO:teuthology.orchestra.run.smithi170.stdout:2719 punch from 0x978e400 to 0x9794200, (0x5e00 bytes) 2024-09-21T23:16:18.852 INFO:teuthology.orchestra.run.smithi170.stdout:2723 read 0x4500200 thru 0x4508a2e (0x882f bytes) 2024-09-21T23:16:18.855 INFO:teuthology.orchestra.run.smithi170.stdout:2726 write 0x52c2000 thru 0x52ce5e2 (0xc5e3 bytes) 2024-09-21T23:16:18.856 INFO:teuthology.orchestra.run.smithi170.stdout:2727 punch from 0x1a72e00 to 0x1a80e00, (0xe000 bytes) 2024-09-21T23:16:18.857 INFO:teuthology.orchestra.run.smithi170.stdout:2729 read 0x8dab600 thru 0x8db359d (0x7f9e bytes) 2024-09-21T23:16:18.881 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:18.880+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:18.888 INFO:teuthology.orchestra.run.smithi170.stdout:2732 trunc from 0x9da5400 to 0xb56200 2024-09-21T23:16:18.928 INFO:teuthology.orchestra.run.smithi170.stdout:2734 punch from 0x37e600 to 0x38d400, (0xee00 bytes) 2024-09-21T23:16:18.965 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:18.965 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:18.965 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:18.965 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:18.965 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:18.968 INFO:teuthology.orchestra.run.smithi170.stdout:2739 write 0x84e3e00 thru 0x84f21f6 (0xe3f7 bytes) 2024-09-21T23:16:18.969 INFO:teuthology.orchestra.run.smithi170.stdout:2740 punch from 0x5e4ce00 to 0x5e55600, (0x8800 bytes) 2024-09-21T23:16:18.970 INFO:teuthology.orchestra.run.smithi170.stdout:2741 punch from 0x2c2b800 to 0x2c2ba00, (0x200 bytes) 2024-09-21T23:16:18.970 INFO:teuthology.orchestra.run.smithi170.stdout:2744 write 0x29eba00 thru 0x29f80cf (0xc6d0 bytes) 2024-09-21T23:16:18.973 INFO:teuthology.orchestra.run.smithi170.stdout:2746 read 0x788e600 thru 0x7897259 (0x8c5a bytes) 2024-09-21T23:16:18.982 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:18.975+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:18.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:18.984 INFO:teuthology.orchestra.run.smithi170.stdout:2748 trunc from 0x84f2200 to 0xc6c3400 2024-09-21T23:16:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:19.006 INFO:teuthology.orchestra.run.smithi170.stdout:2750 read 0xae7b600 thru 0xae8a010 (0xea11 bytes) 2024-09-21T23:16:19.008 INFO:teuthology.orchestra.run.smithi170.stdout:2751 read 0x3f46000 thru 0x3f540ad (0xe0ae bytes) 2024-09-21T23:16:19.010 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:19.021 INFO:teuthology.orchestra.run.smithi170.stdout:2752 trunc from 0xc6c3400 to 0x7f61200 2024-09-21T23:16:19.029 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:19.029 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:19.045 INFO:teuthology.orchestra.run.smithi170.stdout:2754 punch from 0x577b800 to 0x578ac00, (0xf400 bytes) 2024-09-21T23:16:19.046 INFO:teuthology.orchestra.run.smithi170.stdout:2756 write 0x5f34000 thru 0x5f3cfdf (0x8fe0 bytes) 2024-09-21T23:16:19.047 INFO:teuthology.orchestra.run.smithi170.stdout:2760 write 0x3b1b200 thru 0x3b29b64 (0xe965 bytes) 2024-09-21T23:16:19.050 INFO:teuthology.orchestra.run.smithi170.stdout:2763 write 0xee4e00 thru 0xee7664 (0x2865 bytes) 2024-09-21T23:16:19.051 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:19.052 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:19.052 INFO:teuthology.orchestra.run.smithi170.stdout:2765 read 0xf5bc00 thru 0xf66cd1 (0xb0d2 bytes) 2024-09-21T23:16:19.052 INFO:teuthology.orchestra.run.smithi170.stdout:2767 write 0x36b9400 thru 0x36bb770 (0x2371 bytes) 2024-09-21T23:16:19.053 INFO:teuthology.orchestra.run.smithi170.stdout:2771 trunc from 0x7f61200 to 0x82af200 2024-09-21T23:16:19.075 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:19.075 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:19.082 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:19.080+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:19.097 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:19.098 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:19.110 INFO:teuthology.orchestra.run.smithi170.stdout:2772 write 0xd3a8e00 thru 0xd3a9d69 (0xf6a bytes) 2024-09-21T23:16:19.111 INFO:teuthology.orchestra.run.smithi170.stdout:2773 read 0x8625600 thru 0x86312bd (0xbcbe bytes) 2024-09-21T23:16:19.112 INFO:teuthology.orchestra.run.smithi170.stdout:2774 write 0x7cbcc00 thru 0x7cc5681 (0x8a82 bytes) 2024-09-21T23:16:19.112 INFO:teuthology.orchestra.run.smithi170.stdout:2776 read 0x4e38600 thru 0x4e40984 (0x8385 bytes) 2024-09-21T23:16:19.113 INFO:teuthology.orchestra.run.smithi170.stdout:2777 read 0x38bc00 thru 0x39437c (0x877d bytes) 2024-09-21T23:16:19.115 INFO:teuthology.orchestra.run.smithi170.stdout:2781 punch from 0x94f2000 to 0x94f9a00, (0x7a00 bytes) 2024-09-21T23:16:19.115 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:19.115 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:19.116 INFO:teuthology.orchestra.run.smithi170.stdout:2782 clone 17 order 24 su 1048576 sc 12 2024-09-21T23:16:19.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:19.220 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:19.220+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:19.269 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 1] out_osds: [3] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:16:19.269 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:16:19.269 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:16:19.270 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-09-21T23:16:19.270 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T23:16:19.284 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:19.279+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:19.292 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:19.293 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:19.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:19.384 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:19.379+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:19.482 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:19.482 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:19.485 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:19.484+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:19.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:19.587 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:19.583+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:19.662 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:19.662 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:19.676 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:19.722 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:19.719+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:19.760 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T23:16:19.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:16:19.788 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:19.783+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:19.820 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:19.820 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:19.836 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:19.922 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:19.919+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:19.962 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone16 from 0xd3a9e00 (overlap 0xb56200) to 0x911a00 2024-09-21T23:16:19.989 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:19.983+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:20.010 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:20.010 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:20.025 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:20.115 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:20.111+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:20.191 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:20.192+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:20.196 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:20.196 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:20.211 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:20.302 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:20.299+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:20.381 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:20.381 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:20.393 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:20.387+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:20.395 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:20.494 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:20.487+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:20.566 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:20.566 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:20.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:20.627 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:20.623+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:20.695 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:20.691+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:20.754 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:20.754 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:20.763 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:16:20.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:20.807 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-21T23:16:20.810 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:20.807+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:20.897 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:20.896+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:20.936 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:20.936 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:20.950 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:20.998 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:20.996+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:21.067 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:21.067 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:21.081 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:21.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:21.126 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:21.123+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:21.200 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:21.200+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:21.209 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:21.209 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:21.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:21.300 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:21.300+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:21.310 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:21.310 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:21.334 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:21.334 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:21.368 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:21.393 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:21.393 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:21.394 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:21.394 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:21.400 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:21.400 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:21.401 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:21.400+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:21.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:21.424 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:21.424 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:21.449 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:21.449 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:21.511 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:21.512+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:21.541 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:21.541 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:21.541 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:21.541 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:21.541 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:21.555 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:21.644 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:21.644+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:21.704 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:21.699+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:21.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.571992467000001, 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:21.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.506737708999999, 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:16:21.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:16:21.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.109395577000001, 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:16:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.106027888, 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:16:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.101918308, 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:16:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.098125387, 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:16:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0379665800000009, 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:16:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:16:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0342095090000001, 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:16:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:16:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0341827079999995, 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:16:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0341565429999999, 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:16:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0341280929999996, 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:16:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:16:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0340373849999995, 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:16:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:16:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0340058059999997, 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:16:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0339681160000005, 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:16:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:16:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0339377259999996, 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:16:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:16:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0338943090000008, 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:16:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:16:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0338596839999994, 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:16:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:16:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0306061839999998, 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.0305716020000002, 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8148272030000001, 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:16:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:21.831 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:21.832 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:21.832+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:21.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:21.934 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:21.932+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:22.035 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:22.031+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:22.039 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:22.040 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:22.040 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:22.040 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:22.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:22.136 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:22.131+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:22.237 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:22.231+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:22.254 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:22.254 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:22.270 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:22.338 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:22.331+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:22.419 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6568397309999998, 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6567944739999998, 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6567448520000001, 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6567151569999998, 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:22.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:22.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6566822290000003, 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.440 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.444 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6566479330000004, 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:22.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:22.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:22.460 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6566234269999995, 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:22.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:22.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:22.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:22.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.466 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.656595437, 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.471 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6565724089999998, 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.656550899, 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.476 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.656515358, 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.481 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.485 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6564877209999995, 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:22.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:22.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:22.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:22.490 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6564525569999997, 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:22.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:22.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6564175260000003, 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.500 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6563821409999999, 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:22.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:22.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6563419589999997, 2024-09-21T23:16:22.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:22.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6563048309999999, 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:22.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6562647559999997, 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:22.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6562402120000002, 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:22.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:22.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:22.524 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.6562024270000002, 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:22.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:22.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:16:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:16:22.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.529 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:22.529 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:22.529 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:22.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:22.531 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:22.528+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:22.631 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:22.628+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:22.719 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:22.719 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:22.720 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:22.736 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:22.732+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:22.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:22.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:16:22.843 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:22.840+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:22.939 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:22.939 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:22.939 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:22.939 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:22.939 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:22.941 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:22.935+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:22.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:23.036 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:23.031+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:23.135 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:23.131+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:23.187 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:23.187 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:23.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:23.187 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:23.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.251272174999997, 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:23.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:23.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:23.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:23.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:23.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:23.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:23.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.196 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.250251327999997, 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:23.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.250191655000002, 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:23.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:23.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.250101678999997, 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:23.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:23.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:23.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:23.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:23.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:23.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:23.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:23.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:23.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:23.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:23.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.216 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.250051769000002, 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:23.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.218 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:23.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:23.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:23.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:23.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:23.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:23.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:23.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:23.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:23.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.226 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249995628999997, 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:23.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:23.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:23.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:23.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:23.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:23.234 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249941845999999, 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:23.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.242 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:23.242 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.24990957, 2024-09-21T23:16:23.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:23.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.243 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:23.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:23.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:23.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:23.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:23.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:23.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:23.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:23.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.251 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249879290000003, 2024-09-21T23:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:23.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:23.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:23.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.259 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249849634, 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:23.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:23.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:23.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:23.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249819666999997, 2024-09-21T23:16:23.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:23.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:23.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:23.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:23.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249741837999998, 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:23.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:23.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:23.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:23.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:23.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:23.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:23.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:23.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:23.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:23.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:23.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.284 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.284 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:23.284 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249694130999998, 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:23.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.286 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:23.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:23.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:23.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:23.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:23.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:23.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:23.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249663935000001, 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:23.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:23.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:23.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249634018000002, 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.302 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:23.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.302 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:23.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:23.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249604218000002, 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.315 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249574285000001, 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.316 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.317 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:23.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.317 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:23.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:23.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:23.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:23.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:23.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249509322000002, 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:23.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:23.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:23.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249478846000002, 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:23.332 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:23.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:23.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:23.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:23.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:23.338 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.249449681999998, 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:23.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:23.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:23.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.345 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:23.345 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:23.345 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:23.347 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:23.348 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:23.343+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:23.448 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:23.443+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:23.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:23.477 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:23.478 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:23.492 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:23.549 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:23.543+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:23.649 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:23.648+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:23.653 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:23.672 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:23.672 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:23.672 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:23.673 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:23.673 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:23.680 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:23.680 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:23.686 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:23.750 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:23.748+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.594850874000002, 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:16:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:16:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:16:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:16:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.082445590999995, 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:16:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.082380688000001, 2024-09-21T23:16:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:16:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:16:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:16:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:16:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:16:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.082234248999995, 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:16:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:16:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:16:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:16:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.082165133999993, 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:16:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:16:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.081552809000002, 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:16:23.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:16:23.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:16:23.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:16:23.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:16:23.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:16:23.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.874 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.081348403999996, 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.875 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:16:23.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:16:23.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:16:23.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.883 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 97.081305618000002, 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:16:23.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:16:23.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:16:23.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:16:23.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:16:23.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:16:23.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:16:23.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:23.893 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.690347329999994, 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.894 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:23.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:16:23.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:16:23.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:16:23.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:16:23.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:16:23.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:16:23.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.903 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.903 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:23.903 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.690260820000006, 2024-09-21T23:16:23.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:16:23.905 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.905 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.905 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:16:23.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:16:23.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:16:23.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.690183931000007, 2024-09-21T23:16:23.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:16:23.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:23.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:16:23.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:16:23.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:16:23.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:16:23.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:16:23.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:16:23.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:16:23.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689979090999998, 2024-09-21T23:16:23.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:16:23.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:16:23.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:16:23.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:16:23.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:16:23.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:23.931 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689890809000005, 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:23.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:16:23.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:16:23.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:16:23.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:16:23.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:16:23.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689759980999995, 2024-09-21T23:16:23.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:16:23.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.948 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689714327999994, 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.949 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:16:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:16:23.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689675901000001, 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.957 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:16:23.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:16:23.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:16:23.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:23.964 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689546438999997, 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:16:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:16:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:16:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:16:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:16:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:16:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689524797000004, 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:16:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:16:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689504752999994, 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:16:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:16:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:16:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:16:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 96.689420760999994, 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:16:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:16:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:23.996 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:23.997 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:23.997 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:23.899+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:23.998 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:23.998 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:23.998 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:16:23.999 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:23.999 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:24.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:24.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:24.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:24.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:24.013 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:24.007+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:24.110 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:24.107+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:24.140 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:24.140 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:24.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:24.208 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:24.208+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:24.310 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:24.303+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:24.323 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:24.324 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:24.338 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:24.431 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:24.427+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:24.509 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:24.509 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:24.510 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:24.507+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:24.523 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:24.611 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:24.608+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:24.697 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:24.697 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:24.711 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:24.714 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:24.707+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:24.787 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 1] out_osds: [3] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:16:24.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:16:24.814 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:24.807+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:24.837 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:24.837 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:24.853 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:24.915 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:24.912+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:25.015 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:25.016+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:25.026 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:25.027 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:25.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:25.117 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:25.116+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:25.187 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:25.187 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:25.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:25.218 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:25.211+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:25.318 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:25.316+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:25.383 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:25.383 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:25.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:25.419 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:25.415+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:25.520 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:25.515+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:25.566 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:25.566 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:25.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:25.622 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:25.615+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:25.716 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:25.717 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:25.722 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:25.719+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:25.731 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:25.823 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:25.819+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:25.874 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:25.874 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:25.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:25.925 INFO:teuthology.orchestra.run.smithi170.stdout:2783 write 0x2902000 thru 0x290dce7 (0xbce8 bytes) 2024-09-21T23:16:25.927 INFO:teuthology.orchestra.run.smithi170.stdout:2784 read 0xbb96000 thru 0xbb965f7 (0x5f8 bytes) 2024-09-21T23:16:25.928 INFO:teuthology.orchestra.run.smithi170.stdout:2785 punch from 0x6e15a00 to 0x6e22c00, (0xd200 bytes) 2024-09-21T23:16:25.935 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:25.936+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:16:25.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:16:26.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:16:26.012 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:26.025 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:26.019+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:26.026 INFO:teuthology.orchestra.run.smithi170.stdout:2786 trunc from 0xd3a9e00 to 0x888a400 2024-09-21T23:16:26.029 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:26.029 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:26.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:26.087 INFO:teuthology.orchestra.run.smithi170.stdout:2787 punch from 0x1407e00 to 0x1409800, (0x1a00 bytes) 2024-09-21T23:16:26.142 INFO:teuthology.orchestra.run.smithi170.stdout:2792 trunc from 0x888a400 to 0x29dc400 2024-09-21T23:16:26.147 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:26.148+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:26.191 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:26.191 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:26.212 INFO:teuthology.orchestra.run.smithi170.stdout:2793 write 0x3800800 thru 0x3807ac9 (0x72ca bytes) 2024-09-21T23:16:26.225 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:26.226 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:26.227 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:26.223+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:26.248 INFO:teuthology.orchestra.run.smithi170.stdout:2794 write 0x6319a00 thru 0x631dd21 (0x4322 bytes) 2024-09-21T23:16:26.250 INFO:teuthology.orchestra.run.smithi170.stdout:2796 punch from 0x1c08600 to 0x1c09a00, (0x1400 bytes) 2024-09-21T23:16:26.252 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:26.252 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:26.257 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:26.257 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:26.260 INFO:teuthology.orchestra.run.smithi170.stdout:2797 trunc from 0x631de00 to 0xa93da00 2024-09-21T23:16:26.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:26.280 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:26.280 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:26.285 INFO:teuthology.orchestra.run.smithi170.stdout:2803 write 0x54a3000 thru 0x54b2a1d (0xfa1e bytes) 2024-09-21T23:16:26.309 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:26.309 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:26.323 INFO:teuthology.orchestra.run.smithi170.stdout:2806 write 0xd8ae200 thru 0xd8bbe7a (0xdc7b bytes) 2024-09-21T23:16:26.327 INFO:teuthology.orchestra.run.smithi170.stdout:2808 punch from 0x923ea00 to 0x924e800, (0xfe00 bytes) 2024-09-21T23:16:26.327 INFO:teuthology.orchestra.run.smithi170.stdout:2815 write 0x81e3800 thru 0x81e4aea (0x12eb bytes) 2024-09-21T23:16:26.327 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:26.328+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:26.346 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:26.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:26.361 INFO:teuthology.orchestra.run.smithi170.stdout:2816 trunc from 0xd8bc000 to 0x2a5c00 2024-09-21T23:16:26.416 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:26.416 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:26.416 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:26.416 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:26.416 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:26.428 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:26.428+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:26.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:26.455 INFO:teuthology.orchestra.run.smithi170.stdout:2819 punch from 0xf7800 to 0xfb800, (0x4000 bytes) 2024-09-21T23:16:26.458 INFO:teuthology.orchestra.run.smithi170.stdout:2821 punch from 0x84c00 to 0x8b800, (0x6c00 bytes) 2024-09-21T23:16:26.464 INFO:teuthology.orchestra.run.smithi170.stdout:2824 trunc from 0x2a5c00 to 0x2e71200 2024-09-21T23:16:26.519 INFO:teuthology.orchestra.run.smithi170.stdout:2825 write 0x6052e00 thru 0x605b29f (0x84a0 bytes) 2024-09-21T23:16:26.520 INFO:teuthology.orchestra.run.smithi170.stdout:2826 read 0x4655200 thru 0x4657ecc (0x2ccd bytes) 2024-09-21T23:16:26.529 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:26.528+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:26.561 INFO:teuthology.orchestra.run.smithi170.stdout:2827 write 0xcc73e00 thru 0xcc83ad1 (0xfcd2 bytes) 2024-09-21T23:16:26.590 INFO:teuthology.orchestra.run.smithi170.stdout:2828 write 0xd54cc00 thru 0xd54f8c2 (0x2cc3 bytes) 2024-09-21T23:16:26.592 INFO:teuthology.orchestra.run.smithi170.stdout:2832 read 0xa970200 thru 0xa973420 (0x3221 bytes) 2024-09-21T23:16:26.593 INFO:teuthology.orchestra.run.smithi170.stdout:2833 write 0xc309600 thru 0xc30a23b (0xc3c bytes) 2024-09-21T23:16:26.593 INFO:teuthology.orchestra.run.smithi170.stdout:2834 write 0xa849000 thru 0xa84e4f6 (0x54f7 bytes) 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.672164422000002, 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:26.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:26.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671143575000002, 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:26.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:26.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671083901999999, 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:26.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670993926000001, 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:26.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:26.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:26.622 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670944016, 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:26.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.627 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670887876000002, 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:26.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:26.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670834093000003, 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.633 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:26.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670801816999997, 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:26.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:26.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670771537, 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:26.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670741880999998, 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670711914000002, 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670634085000003, 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670586378000003, 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670556181999999, 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670526264999999, 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670496464999999, 2024-09-21T23:16:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670466531999999, 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670401568999999, 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670371093, 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.670341929000003, 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:26.705 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:26.707 INFO:teuthology.orchestra.run.smithi170.stdout:2837 trunc from 0xd54fa00 to 0x72a0800 2024-09-21T23:16:26.707 INFO:teuthology.orchestra.run.smithi170.stdout:2838 read 0x6143a00 thru 0x615110f (0xd710 bytes) 2024-09-21T23:16:26.707 INFO:teuthology.orchestra.run.smithi170.stdout:2840 write 0x8e16a00 thru 0x8e24aec (0xe0ed bytes) 2024-09-21T23:16:26.707 INFO:teuthology.orchestra.run.smithi170.stdout:2841 trunc from 0x8e24c00 to 0x94a2a00 2024-09-21T23:16:26.707 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:26.708 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:26.703+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:26.731 INFO:teuthology.orchestra.run.smithi170.stdout:2842 write 0x9842600 thru 0x984e9fc (0xc3fd bytes) 2024-09-21T23:16:26.733 INFO:teuthology.orchestra.run.smithi170.stdout:2846 read 0x2249600 thru 0x224a35b (0xd5c bytes) 2024-09-21T23:16:26.743 INFO:teuthology.orchestra.run.smithi170.stdout:2847 trunc from 0x984ea00 to 0x56b4200 2024-09-21T23:16:26.783 INFO:teuthology.orchestra.run.smithi170.stdout:2850 trunc from 0x56b4200 to 0xa5e6800 2024-09-21T23:16:26.802 INFO:teuthology.orchestra.run.smithi170.stdout:2851 punch from 0x5d52200 to 0x5d55a00, (0x3800 bytes) 2024-09-21T23:16:26.803 INFO:teuthology.orchestra.run.smithi170.stdout:2852 write 0x934b200 thru 0x9352244 (0x7045 bytes) 2024-09-21T23:16:26.807 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:26.808+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:26.811 INFO:teuthology.orchestra.run.smithi170.stdout:2854 trunc from 0xa5e6800 to 0x6238200 2024-09-21T23:16:26.848 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:26.848 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:26.849 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:26.849 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:26.849 INFO:teuthology.orchestra.run.smithi170.stdout:2855 write 0x26f0600 thru 0x26f8b3c (0x853d bytes) 2024-09-21T23:16:26.863 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:26.878 INFO:teuthology.orchestra.run.smithi170.stdout:2857 write 0x7600400 thru 0x7606a73 (0x6674 bytes) 2024-09-21T23:16:26.879 INFO:teuthology.orchestra.run.smithi170.stdout:2858 read 0x75d8000 thru 0x75d97c8 (0x17c9 bytes) 2024-09-21T23:16:26.909 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:26.908+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:26.915 INFO:teuthology.orchestra.run.smithi170.stdout:2860 write 0xc5d4600 thru 0xc5e2870 (0xe271 bytes) 2024-09-21T23:16:26.918 INFO:teuthology.orchestra.run.smithi170.stdout:2867 punch from 0x8233800 to 0x823e800, (0xb000 bytes) 2024-09-21T23:16:26.936 INFO:teuthology.orchestra.run.smithi170.stdout:2870 trunc from 0xc5e2a00 to 0x4db7a00 2024-09-21T23:16:26.956 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:16:26.961 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:16:26.988 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:16:26.988 INFO:teuthology.orchestra.run.smithi170.stdout:2871 read 0x4a6800 thru 0x4b25df (0xbde0 bytes) 2024-09-21T23:16:26.991 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:26.991 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:26.991 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:26.991 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:26.991 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:26.992 INFO:teuthology.orchestra.run.smithi170.stdout:2872 punch from 0x1bc3800 to 0x1bcc600, (0x8e00 bytes) 2024-09-21T23:16:26.992 INFO:teuthology.orchestra.run.smithi170.stdout:2874 read 0x230400 thru 0x23114b (0xd4c bytes) 2024-09-21T23:16:27.004 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:27.022 INFO:teuthology.orchestra.run.smithi170.stdout:2876 write 0xd3abe00 thru 0xd3baebd (0xf0be bytes) 2024-09-21T23:16:27.024 INFO:teuthology.orchestra.run.smithi170.stdout:2877 write 0x2debe00 thru 0x2df8e39 (0xd03a bytes) 2024-09-21T23:16:27.025 INFO:teuthology.orchestra.run.smithi170.stdout:2878 write 0x4ad4800 thru 0x4ae0e11 (0xc612 bytes) 2024-09-21T23:16:27.026 INFO:teuthology.orchestra.run.smithi170.stdout:2881 write 0x86ac00 thru 0x86b483 (0x884 bytes) 2024-09-21T23:16:27.032 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:27.032+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:27.037 INFO:teuthology.orchestra.run.smithi170.stdout:2887 trunc from 0xd3bb000 to 0x957e600 2024-09-21T23:16:27.064 INFO:teuthology.orchestra.run.smithi170.stdout:2888 read 0x2a64a00 thru 0x2a68729 (0x3d2a bytes) 2024-09-21T23:16:27.071 INFO:teuthology.orchestra.run.smithi170.stdout:2889 trunc from 0x957e600 to 0x6390400 2024-09-21T23:16:27.088 INFO:teuthology.orchestra.run.smithi170.stdout:2890 write 0x1323a00 thru 0x1329fac (0x65ad bytes) 2024-09-21T23:16:27.110 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:27.108+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:27.112 INFO:teuthology.orchestra.run.smithi170.stdout:2893 write 0x837d000 thru 0x838a853 (0xd854 bytes) 2024-09-21T23:16:27.115 INFO:teuthology.orchestra.run.smithi170.stdout:2894 trunc from 0x838aa00 to 0x7837400 2024-09-21T23:16:27.137 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.915836834, 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:16:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.403431551, 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:16:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:16:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.403366648, 2024-09-21T23:16:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:16:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.403220209, 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:16:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:16:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.40315109399999, 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:16:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:16:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.402538769, 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:16:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.402334364, 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:16:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:16:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.402291578, 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:16:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:16:27.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.188 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.01133329, 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.189 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:16:27.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:16:27.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.194 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.01124677999999, 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:16:27.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:16:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.01116989099999, 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:16:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:16:27.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.010965051, 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:16:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.212 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.01087676900001, 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:16:27.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:16:27.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.218 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.010745941, 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:16:27.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.224 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.010700288, 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:27.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:16:27.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:16:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:27.230 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.010661861, 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:27.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.010532399, 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:16:27.236 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:27.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:16:27.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.01051075700001, 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.242 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:16:27.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.010490713, 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.248 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:16:27.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:16:27.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 100.010406721, 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:16:27.254 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:16:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:16:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:27.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.260 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:27.262 INFO:teuthology.orchestra.run.smithi170.stdout:2897 write 0xa028a00 thru 0xa032cfa (0xa2fb bytes) 2024-09-21T23:16:27.262 INFO:teuthology.orchestra.run.smithi170.stdout:2899 trunc from 0xa032e00 to 0x7016a00 2024-09-21T23:16:27.262 INFO:teuthology.orchestra.run.smithi170.stdout:2900 trunc from 0x7016a00 to 0x7240a00 2024-09-21T23:16:27.262 INFO:teuthology.orchestra.run.smithi170.stdout:2902 read 0xd96c00 thru 0xd9b47b (0x487c bytes) 2024-09-21T23:16:27.262 INFO:teuthology.orchestra.run.smithi170.stdout:2903 trunc from 0x7240a00 to 0x4f70200 2024-09-21T23:16:27.262 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:27.263 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:27.260+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:27.275 INFO:teuthology.orchestra.run.smithi170.stdout:2905 write 0xe1f7800 thru 0xe20613f (0xe940 bytes) 2024-09-21T23:16:27.301 INFO:teuthology.orchestra.run.smithi170.stdout:2906 trunc from 0xe206200 to 0x132a200 2024-09-21T23:16:27.362 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:27.359+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:27.400 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:27.401 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:27.401 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:27.401 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:27.403 INFO:teuthology.orchestra.run.smithi170.stdout:2908 write 0x86e4000 thru 0x86e8fa3 (0x4fa4 bytes) 2024-09-21T23:16:27.406 INFO:teuthology.orchestra.run.smithi170.stdout:2911 write 0x2bda400 thru 0x2be07f2 (0x63f3 bytes) 2024-09-21T23:16:27.407 INFO:teuthology.orchestra.run.smithi170.stdout:2912 read 0x412c00 thru 0x4159d0 (0x2dd1 bytes) 2024-09-21T23:16:27.408 INFO:teuthology.orchestra.run.smithi170.stdout:2913 read 0x71ba800 thru 0x71c40db (0x98dc bytes) 2024-09-21T23:16:27.415 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:27.437 INFO:teuthology.orchestra.run.smithi170.stdout:2915 write 0xaa49200 thru 0xaa4bfa8 (0x2da9 bytes) 2024-09-21T23:16:27.445 INFO:teuthology.orchestra.run.smithi170.stdout:2916 trunc from 0xaa4c000 to 0x79dee00 2024-09-21T23:16:27.479 INFO:teuthology.orchestra.run.smithi170.stdout:2919 read 0x24bb400 thru 0x24bfd9a (0x499b bytes) 2024-09-21T23:16:27.485 INFO:teuthology.orchestra.run.smithi170.stdout:2920 write 0x3a6bc00 thru 0x3a701a2 (0x45a3 bytes) 2024-09-21T23:16:27.485 INFO:teuthology.orchestra.run.smithi170.stdout:2923 clone 18 order 19 su 4096 sc 11 2024-09-21T23:16:27.502 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:27.499+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:27.564 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:27.564+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:27.588 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:27.588 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:27.589 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:27.589 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:27.589 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:27.602 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:27.665 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:27.664+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:27.766 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:27.764+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.813627425999996, 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:16:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:16:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.813360441999997, 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:16:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.201031897, 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:16:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:16:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.788 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.200989198, 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:27.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.200950578, 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:16:27.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.200918706, 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:16:27.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.200885482, 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:16:27.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.200850407000001, 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.810 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.200805449000001, 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:16:27.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:16:27.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.200353381999999, 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:16:27.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.199638903, 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:16:27.819 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:16:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.199597833, 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:16:27.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.199140715, 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.198904036, 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.832 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.198488455, 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:16:27.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:16:27.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.197810795000001, 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:16:27.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:16:27.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.197319505999999, 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.845 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.849 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.197062608, 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26034 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7325696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.196599527, 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086107304999999995, 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26034 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571018+0000", 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018324500000000001 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571023+0000", 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9009999999999999e-06 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571048+0000", 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4454e-05 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571058+0000", 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0145e-05 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656654+0000", 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085595579000000005 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656758+0000", 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010446999999999999 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656942+0000", 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183511 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26035 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6836224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:27.858 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 11.196359142, 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086157396999999997, 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26035 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571805+0000", 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000730326 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571812+0000", 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571833+0000", 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1506999999999999e-05 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571846+0000", 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2330999999999999e-05 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657009+0000", 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085162977000000001 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657091+0000", 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2514000000000003e-05 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657232+0000", 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140858 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.862 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.863 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.863 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.863 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.863 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:27.864 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:27.867 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:27.864+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:27.963 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:16:27.968 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:27.968+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:27.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:16:27.992 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:11034 3.f 3:f1121bf4:::rbd_header.10f1263c9d53:head [call rbd.snapshot_get in=23b out=52b,call rbd.parent_overlap_get in=29b out=9b,call rbd.get_flags in=20b out=8b,call rbd.get_protection_status in=32b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:27.990612+0000", 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00075646399999999999, 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00084745600000000001, 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:27.993 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 11034 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990612+0000", 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990612+0000", 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990614+0000", 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990627+0000", 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3203e-05 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990648+0000", 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0941000000000002e-05 2024-09-21T23:16:27.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990660+0000", 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1933000000000001e-05 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990740+0000", 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0442e-05 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:27.990863+0000", 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000123153 2024-09-21T23:16:27.996 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.997 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:27.997 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.997 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:27.997 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:16:27.997 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:16:27.997 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:28.008 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:28.069 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:28.063+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:28.132 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:28.132 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:28.133 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:28.133 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:28.133 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:28.146 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:28.170 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:28.168+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:28.194 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone17 from 0x79dee00 (overlap 0x2a5c00) to 0x10f000 2024-09-21T23:16:28.271 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:28.268+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.1984754, 2024-09-21T23:16:28.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:28.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:16:28.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10124 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6156288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.194117761999999, 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038330863999999999, 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10124 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0399999999999995e-07 2024-09-21T23:16:28.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125818+0000", 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125827+0000", 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6330000000000002e-06 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125834+0000", 2024-09-21T23:16:28.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9199999999999998e-06 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.163896+0000", 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038062072000000002 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164009+0000", 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011386400000000001 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164145+0000", 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013601100000000001 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10125 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6141952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.194079392000001, 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038589275999999999, 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.335 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10125 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125854+0000", 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125856+0000", 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8159999999999998e-06 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125865+0000", 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7739999999999993e-06 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125872+0000", 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164189+0000", 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038317276999999997 2024-09-21T23:16:28.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164304+0000", 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114676 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164442+0000", 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013814 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10126 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6172672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.194051609000001, 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038865093000000003, 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10126 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.340 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:28.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125884+0000", 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125893+0000", 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125900+0000", 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0500000000000003e-06 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164491+0000", 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038591473000000001 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164606+0000", 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115109 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164746+0000", 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013963800000000001 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.194024242999999, 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.344 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:16:28.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:16:28.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:16:28.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.19399677, 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:16:28.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:16:28.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193956597, 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:16:28.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:16:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193905828, 2024-09-21T23:16:28.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:16:28.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:16:28.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193875244999999, 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.363 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:16:28.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193847339, 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.368 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:16:28.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193820119, 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.372 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:16:28.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:16:28.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.1937932, 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:16:28.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:16:28.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193737436999999, 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:16:28.381 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193704264000001, 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:16:28.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:16:28.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193659045, 2024-09-21T23:16:28.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:28.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:16:28.394 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193619764999999, 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.395 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:16:28.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.399 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193589278999999, 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:28.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193559986, 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.404 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.408 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193517644, 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:16:28.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:16:28.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.193134449, 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:28.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:28.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:28.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:28.417 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:28.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:28.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:28.419 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:28.416+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:28.519 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:28.516+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:28.558 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:28.558 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:28.558 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:28.558 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:28.567 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:28.568 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:28.572 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:28.585 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:28.585 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:28.602 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:28.620 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:28.616+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:28.623 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:28.623 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:28.646 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:28.670 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:28.671 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:28.704 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:28.704 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:28.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:28.760 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:28.760+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:28.822 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:28.816+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:28.885 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:28.885 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:28.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:28.923 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:28.920+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:28.968 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:16:28.990 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:16:28.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:16:29.023 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:29.020+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:29.027 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:29.027 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:29.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:29.124 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:29.120+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:29.140 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-21T23:16:29.221 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:29.222 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:29.225 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:29.220+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:29.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:29.292 INFO:teuthology.orchestra.run.smithi170.stdout:2924 read 0x360f800 thru 0x361d0c1 (0xd8c2 bytes) 2024-09-21T23:16:29.294 INFO:teuthology.orchestra.run.smithi170.stdout:2927 read 0x6060c00 thru 0x6068cf6 (0x80f7 bytes) 2024-09-21T23:16:29.298 INFO:teuthology.orchestra.run.smithi170.stdout:2930 read 0x75f3a00 thru 0x75f9620 (0x5c21 bytes) 2024-09-21T23:16:29.302 INFO:teuthology.orchestra.run.smithi170.stdout:2931 read 0x296b000 thru 0x29758f1 (0xa8f2 bytes) 2024-09-21T23:16:29.325 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:29.320+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:29.327 INFO:teuthology.orchestra.run.smithi170.stdout:2934 write 0x7e04400 thru 0x7e13833 (0xf434 bytes) 2024-09-21T23:16:29.328 INFO:teuthology.orchestra.run.smithi170.stdout:2935 punch from 0x1bdd800 to 0x1be3400, (0x5c00 bytes) 2024-09-21T23:16:29.356 INFO:teuthology.orchestra.run.smithi170.stdout:2936 write 0x28f2e00 thru 0x28fa8b3 (0x7ab4 bytes) 2024-09-21T23:16:29.371 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:29.371 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:29.381 INFO:teuthology.orchestra.run.smithi170.stdout:2938 read 0x45bd400 thru 0x45c5e34 (0x8a35 bytes) 2024-09-21T23:16:29.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:29.431 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:29.432+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:29.509 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:29.509 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:29.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:29.568 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:29.568+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:29.628 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:29.628+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:29.681 INFO:teuthology.orchestra.run.smithi170.stdout:2939 write 0x7e9b000 thru 0x7ea55ab (0xa5ac bytes) 2024-09-21T23:16:29.684 INFO:teuthology.orchestra.run.smithi170.stdout:2940 read 0x5d48200 thru 0x5d48e3c (0xc3d bytes) 2024-09-21T23:16:29.715 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:29.715 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:29.729 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:29.724+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:29.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:29.741 INFO:teuthology.orchestra.run.smithi170.stdout:2941 write 0x9795e00 thru 0x97a1b65 (0xbd66 bytes) 2024-09-21T23:16:29.744 INFO:teuthology.orchestra.run.smithi170.stdout:2942 write 0x79b3c00 thru 0x79beca7 (0xb0a8 bytes) 2024-09-21T23:16:29.744 INFO:teuthology.orchestra.run.smithi170.stdout:2943 read 0x303ae00 thru 0x304a9d5 (0xfbd6 bytes) 2024-09-21T23:16:29.747 INFO:teuthology.orchestra.run.smithi170.stdout:2944 read 0xa77800 thru 0xa79e33 (0x2634 bytes) 2024-09-21T23:16:29.749 INFO:teuthology.orchestra.run.smithi170.stdout:2945 read 0x6443c00 thru 0x64465b0 (0x29b1 bytes) 2024-09-21T23:16:29.792 INFO:teuthology.orchestra.run.smithi170.stdout:2946 trunc from 0x97a1c00 to 0x53dd400 2024-09-21T23:16:29.829 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:29.824+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:29.930 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:29.928+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:29.942 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:29.942 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:29.959 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:29.974 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:16:29.977 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:16:29.989 INFO:teuthology.orchestra.run.smithi170.stdout:2947 trunc from 0x53dd400 to 0xb476600 2024-09-21T23:16:30.002 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:16:30.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:16:30.008 INFO:teuthology.orchestra.run.smithi170.stdout:2949 read 0x410ce00 thru 0x41124a9 (0x56aa bytes) 2024-09-21T23:16:30.011 INFO:teuthology.orchestra.run.smithi170.stdout:2952 punch from 0xaaeea00 to 0xaaf4e00, (0x6400 bytes) 2024-09-21T23:16:30.012 INFO:teuthology.orchestra.run.smithi170.stdout:2955 write 0xa655a00 thru 0xa665348 (0xf949 bytes) 2024-09-21T23:16:30.052 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:30.052+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:30.075 INFO:teuthology.orchestra.run.smithi170.stdout:2956 trunc from 0xb476600 to 0x1670600 2024-09-21T23:16:30.093 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:30.093 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:30.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:30.133 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:30.132+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:30.212 INFO:teuthology.orchestra.run.smithi170.stdout:2958 write 0x14f0600 thru 0x14f452c (0x3f2d bytes) 2024-09-21T23:16:30.216 INFO:teuthology.orchestra.run.smithi170.stdout:2959 read 0x15cde00 thru 0x15d9a36 (0xbc37 bytes) 2024-09-21T23:16:30.220 INFO:teuthology.orchestra.run.smithi170.stdout:2960 punch from 0xd9c600 to 0xd9e800, (0x2200 bytes) 2024-09-21T23:16:30.234 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:30.232+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:30.253 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:30.253 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:30.269 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:30.276 INFO:teuthology.orchestra.run.smithi170.stdout:2962 write 0x688b600 thru 0x68922a7 (0x6ca8 bytes) 2024-09-21T23:16:30.328 INFO:teuthology.orchestra.run.smithi170.stdout:2963 write 0xc779e00 thru 0xc77ea9b (0x4c9c bytes) 2024-09-21T23:16:30.331 INFO:teuthology.orchestra.run.smithi170.stdout:2965 write 0x2e16400 thru 0x2e1a008 (0x3c09 bytes) 2024-09-21T23:16:30.331 INFO:teuthology.orchestra.run.smithi170.stdout:2968 write 0x6ee7600 thru 0x6ee989c (0x229d bytes) 2024-09-21T23:16:30.331 INFO:teuthology.orchestra.run.smithi170.stdout:2969 read 0x371a800 thru 0x3726090 (0xb891 bytes) 2024-09-21T23:16:30.332 INFO:teuthology.orchestra.run.smithi170.stdout:2971 write 0xa14e400 thru 0xa15cd8a (0xe98b bytes) 2024-09-21T23:16:30.332 INFO:teuthology.orchestra.run.smithi170.stdout:2972 write 0x24c3200 thru 0x24d07fd (0xd5fe bytes) 2024-09-21T23:16:30.332 INFO:teuthology.orchestra.run.smithi170.stdout:2973 read 0x8507400 thru 0x850db4f (0x6750 bytes) 2024-09-21T23:16:30.333 INFO:teuthology.orchestra.run.smithi170.stdout:2975 trunc from 0xc77ec00 to 0xc106a00 2024-09-21T23:16:30.335 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:30.332+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:30.391 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:30.392 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:30.394 INFO:teuthology.orchestra.run.smithi170.stdout:2976 read 0x98e2a00 thru 0x98eea57 (0xc058 bytes) 2024-09-21T23:16:30.395 INFO:teuthology.orchestra.run.smithi170.stdout:2977 punch from 0xab8ea00 to 0xab9c200, (0xd800 bytes) 2024-09-21T23:16:30.396 INFO:teuthology.orchestra.run.smithi170.stdout:2979 read 0xb4ffa00 thru 0xb5012d6 (0x18d7 bytes) 2024-09-21T23:16:30.396 INFO:teuthology.orchestra.run.smithi170.stdout:2980 read 0x74be400 thru 0x74cb22f (0xce30 bytes) 2024-09-21T23:16:30.396 INFO:teuthology.orchestra.run.smithi170.stdout:2981 write 0x853d000 thru 0x8549eb3 (0xceb4 bytes) 2024-09-21T23:16:30.397 INFO:teuthology.orchestra.run.smithi170.stdout:2982 punch from 0xbc58800 to 0xbc61000, (0x8800 bytes) 2024-09-21T23:16:30.398 INFO:teuthology.orchestra.run.smithi170.stdout:2983 read 0xa24ca00 thru 0xa255721 (0x8d22 bytes) 2024-09-21T23:16:30.406 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:30.418 INFO:teuthology.orchestra.run.smithi170.stdout:2984 trunc from 0xc106a00 to 0x28a7000 2024-09-21T23:16:30.436 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:30.436+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:30.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:16:30.489 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-21T23:16:29.975218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336078850,"num_objects":763,"num_object_clones":174,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":348,"num_read":161523,"num_read_kb":422832,"num_write":10854,"num_write_kb":67379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1409,"num_bytes_recovered":17081594,"num_keys_recovered":199,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9619,"ondisk_log_size":9619,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":255736,"kb_used_data":50056,"kb_used_omap":11,"kb_used_meta":205556,"kb_avail":471603464,"statfs":{"total":483183820800,"available":482921947136,"internally_reserved":0,"allocated":51257344,"data_stored":58487096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12231,"internal_metadata":210489401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31761931,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":5,"num_read":20853,"num_read_kb":28996,"num_write":437,"num_write_kb":2665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.034685"},"pg_stats":[{"pgid":"3.1f","version":"79'268","reported_seq":9065,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.113749+0000","last_change":"2024-09-21T23:16:06.209669+0000","last_active":"2024-09-21T23:16:26.113749+0000","last_peered":"2024-09-21T23:16:26.113749+0000","last_clean":"2024-09-21T23:16:26.113749+0000","last_became_active":"2024-09-21T23:16:06.208917+0000","last_became_peered":"2024-09-21T23:16:06.208917+0000","last_unstale":"2024-09-21T23:16:26.113749+0000","last_undegraded":"2024-09-21T23:16:26.113749+0000","last_fullsized":"2024-09-21T23:16:26.113749+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:11:21.474520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":10777177,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":9327,"num_read_kb":9167,"num_write":356,"num_write_kb":822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":14388,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"82'202","reported_seq":1207,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.826524+0000","last_change":"2024-09-21T23:16:18.771067+0000","last_active":"2024-09-21T23:16:29.826524+0000","last_peered":"2024-09-21T23:16:29.826524+0000","last_clean":"2024-09-21T23:16:29.826524+0000","last_became_active":"2024-09-21T23:16:18.768784+0000","last_became_peered":"2024-09-21T23:16:18.768784+0000","last_unstale":"2024-09-21T23:16:29.826524+0000","last_undegraded":"2024-09-21T23:16:29.826524+0000","last_fullsized":"2024-09-21T23:16:29.826524+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:51:53.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":9015860,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":738,"num_read_kb":9336,"num_write":189,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2209332,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"82'102","reported_seq":1366,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.719645+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:16:29.719645+0000","last_peered":"2024-09-21T23:16:29.719645+0000","last_clean":"2024-09-21T23:16:29.719645+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:16:29.719645+0000","last_undegraded":"2024-09-21T23:16:29.719645+0000","last_fullsized":"2024-09-21T23:16:29.719645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:51:33.292362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":4909568,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":1143,"num_read_kb":6797,"num_write":97,"num_write_kb":2013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"81'267","reported_seq":3417,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.881351+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:16:29.881351+0000","last_peered":"2024-09-21T23:16:29.881351+0000","last_clean":"2024-09-21T23:16:29.881351+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:16:29.881351+0000","last_undegraded":"2024-09-21T23:16:29.881351+0000","last_fullsized":"2024-09-21T23:16:29.881351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:18:44.303903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":16168528,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":3835,"num_read_kb":17775,"num_write":239,"num_write_kb":10571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"68'244","reported_seq":2081,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.961873+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:16:25.961873+0000","last_peered":"2024-09-21T23:16:25.961873+0000","last_clean":"2024-09-21T23:16:25.961873+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:16:25.961873+0000","last_undegraded":"2024-09-21T23:16:25.961873+0000","last_fullsized":"2024-09-21T23:16:25.961873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:43:24.435448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":5896734,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":2700,"num_read_kb":11597,"num_write":259,"num_write_kb":876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"82'302","reported_seq":5694,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.740041+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:16:29.740041+0000","last_peered":"2024-09-21T23:16:29.740041+0000","last_clean":"2024-09-21T23:16:29.740041+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:16:29.740041+0000","last_undegraded":"2024-09-21T23:16:29.740041+0000","last_fullsized":"2024-09-21T23:16:29.740041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:04:38.638877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":6019619,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":6947,"num_read_kb":16872,"num_write":222,"num_write_kb":794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"82'142","reported_seq":1278,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.800520+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:16:29.800520+0000","last_peered":"2024-09-21T23:16:29.800520+0000","last_clean":"2024-09-21T23:16:29.800520+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:16:29.800520+0000","last_undegraded":"2024-09-21T23:16:29.800520+0000","last_fullsized":"2024-09-21T23:16:29.800520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:11.479193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":8011792,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":969,"num_read_kb":6565,"num_write":132,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"82'376","reported_seq":10673,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.820617+0000","last_change":"2024-09-21T23:16:06.209284+0000","last_active":"2024-09-21T23:16:29.820617+0000","last_peered":"2024-09-21T23:16:29.820617+0000","last_clean":"2024-09-21T23:16:29.820617+0000","last_became_active":"2024-09-21T23:16:06.208604+0000","last_became_peered":"2024-09-21T23:16:06.208604+0000","last_unstale":"2024-09-21T23:16:29.820617+0000","last_undegraded":"2024-09-21T23:16:29.820617+0000","last_fullsized":"2024-09-21T23:16:29.820617+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:45.657512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":24008252,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":11223,"num_read_kb":24431,"num_write":523,"num_write_kb":8284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":75324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"68'170","reported_seq":8903,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.961964+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:16:25.961964+0000","last_peered":"2024-09-21T23:16:25.961964+0000","last_clean":"2024-09-21T23:16:25.961964+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:16:25.961964+0000","last_undegraded":"2024-09-21T23:16:25.961964+0000","last_fullsized":"2024-09-21T23:16:25.961964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:58.546596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":11476032,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":8592,"num_read_kb":15658,"num_write":157,"num_write_kb":683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"82'351","reported_seq":2059,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.881329+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:16:29.881329+0000","last_peered":"2024-09-21T23:16:29.881329+0000","last_clean":"2024-09-21T23:16:29.881329+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:16:29.881329+0000","last_undegraded":"2024-09-21T23:16:29.881329+0000","last_fullsized":"2024-09-21T23:16:29.881329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:09:58.769914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":31935138,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":30,"num_read":2158,"num_read_kb":7024,"num_write":438,"num_write_kb":2627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"69'122","reported_seq":2954,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.962119+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:16:25.962119+0000","last_peered":"2024-09-21T23:16:25.962119+0000","last_clean":"2024-09-21T23:16:25.962119+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:16:25.962119+0000","last_undegraded":"2024-09-21T23:16:25.962119+0000","last_fullsized":"2024-09-21T23:16:25.962119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:11:29.668035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898304,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":2713,"num_read_kb":24295,"num_write":120,"num_write_kb":2627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"82'497","reported_seq":1858,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.805335+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:16:29.805335+0000","last_peered":"2024-09-21T23:16:29.805335+0000","last_clean":"2024-09-21T23:16:29.805335+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:16:29.805335+0000","last_undegraded":"2024-09-21T23:16:29.805335+0000","last_fullsized":"2024-09-21T23:16:29.805335+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:42.653598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8018079,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":3022,"num_read_kb":11682,"num_write":736,"num_write_kb":1407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.036285+0000","last_change":"2024-09-21T23:15:06.350164+0000","last_active":"2024-09-21T23:16:29.036285+0000","last_peered":"2024-09-21T23:16:29.036285+0000","last_clean":"2024-09-21T23:16:29.036285+0000","last_became_active":"2024-09-21T23:15:06.349271+0000","last_became_peered":"2024-09-21T23:15:06.349271+0000","last_unstale":"2024-09-21T23:16:29.036285+0000","last_undegraded":"2024-09-21T23:16:29.036285+0000","last_fullsized":"2024-09-21T23:16:29.036285+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:56.638782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'174","reported_seq":17156,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.048887+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:16:26.048887+0000","last_peered":"2024-09-21T23:16:26.048887+0000","last_clean":"2024-09-21T23:16:26.048887+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:16:26.048887+0000","last_undegraded":"2024-09-21T23:16:26.048887+0000","last_fullsized":"2024-09-21T23:16:26.048887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:59:22.879499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":25137192,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":16534,"num_read_kb":24043,"num_write":164,"num_write_kb":2086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.021456+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:16:29.021456+0000","last_peered":"2024-09-21T23:16:29.021456+0000","last_clean":"2024-09-21T23:16:29.021456+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:16:29.021456+0000","last_undegraded":"2024-09-21T23:16:29.021456+0000","last_fullsized":"2024-09-21T23:16:29.021456+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"82'1475","reported_seq":4360,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:28.974011+0000","last_change":"2024-09-21T23:16:18.776075+0000","last_active":"2024-09-21T23:16:28.974011+0000","last_peered":"2024-09-21T23:16:28.974011+0000","last_clean":"2024-09-21T23:16:28.974011+0000","last_became_active":"2024-09-21T23:16:18.771731+0000","last_became_peered":"2024-09-21T23:16:18.771731+0000","last_unstale":"2024-09-21T23:16:28.974011+0000","last_undegraded":"2024-09-21T23:16:28.974011+0000","last_fullsized":"2024-09-21T23:16:28.974011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1475,"log_dups_size":0,"ondisk_log_size":1475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:42:09.391659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":8584106,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":9811,"num_read_kb":29148,"num_write":2652,"num_write_kb":5385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":2243370,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.961969+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:16:25.961969+0000","last_peered":"2024-09-21T23:16:25.961969+0000","last_clean":"2024-09-21T23:16:25.961969+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:16:25.961969+0000","last_undegraded":"2024-09-21T23:16:25.961969+0000","last_fullsized":"2024-09-21T23:16:25.961969+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"82'257","reported_seq":997,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.820494+0000","last_change":"2024-09-21T23:16:17.766149+0000","last_active":"2024-09-21T23:16:29.820494+0000","last_peered":"2024-09-21T23:16:29.820494+0000","last_clean":"2024-09-21T23:16:29.820494+0000","last_became_active":"2024-09-21T23:16:17.765404+0000","last_became_peered":"2024-09-21T23:16:17.765404+0000","last_unstale":"2024-09-21T23:16:29.820494+0000","last_undegraded":"2024-09-21T23:16:29.820494+0000","last_fullsized":"2024-09-21T23:16:29.820494+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:50:03.304250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":5189155,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1609,"num_read_kb":2627,"num_write":305,"num_write_kb":765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":113664,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.021291+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:16:29.021291+0000","last_peered":"2024-09-21T23:16:29.021291+0000","last_clean":"2024-09-21T23:16:29.021291+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:16:29.021291+0000","last_undegraded":"2024-09-21T23:16:29.021291+0000","last_fullsized":"2024-09-21T23:16:29.021291+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"82'294","reported_seq":2455,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.796392+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:16:29.796392+0000","last_peered":"2024-09-21T23:16:29.796392+0000","last_clean":"2024-09-21T23:16:29.796392+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:16:29.796392+0000","last_undegraded":"2024-09-21T23:16:29.796392+0000","last_fullsized":"2024-09-21T23:16:29.796392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:02:11.655743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":10286407,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3391,"num_read_kb":14958,"num_write":338,"num_write_kb":1526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'208","reported_seq":2354,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.802934+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:16:29.802934+0000","last_peered":"2024-09-21T23:16:29.802934+0000","last_clean":"2024-09-21T23:16:29.802934+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:16:29.802934+0000","last_undegraded":"2024-09-21T23:16:29.802934+0000","last_fullsized":"2024-09-21T23:16:29.802934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:22:27.188859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":11370014,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":2056,"num_read_kb":12156,"num_write":209,"num_write_kb":853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.022547+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:16:29.022547+0000","last_peered":"2024-09-21T23:16:29.022547+0000","last_clean":"2024-09-21T23:16:29.022547+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:16:29.022547+0000","last_undegraded":"2024-09-21T23:16:29.022547+0000","last_fullsized":"2024-09-21T23:16:29.022547+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"82'213","reported_seq":2786,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.824090+0000","last_change":"2024-09-21T23:16:18.772875+0000","last_active":"2024-09-21T23:16:29.824090+0000","last_peered":"2024-09-21T23:16:29.824090+0000","last_clean":"2024-09-21T23:16:29.824090+0000","last_became_active":"2024-09-21T23:16:18.770556+0000","last_became_peered":"2024-09-21T23:16:18.770556+0000","last_unstale":"2024-09-21T23:16:29.824090+0000","last_undegraded":"2024-09-21T23:16:29.824090+0000","last_fullsized":"2024-09-21T23:16:29.824090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:12:10.034857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":3751540,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":2452,"num_read_kb":7963,"num_write":179,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":80516,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:28.973269+0000","last_change":"2024-09-21T23:16:15.257881+0000","last_active":"2024-09-21T23:16:28.973269+0000","last_peered":"2024-09-21T23:16:28.973269+0000","last_clean":"2024-09-21T23:16:28.973269+0000","last_became_active":"2024-09-21T23:16:15.257496+0000","last_became_peered":"2024-09-21T23:16:15.257496+0000","last_unstale":"2024-09-21T23:16:28.973269+0000","last_undegraded":"2024-09-21T23:16:28.973269+0000","last_fullsized":"2024-09-21T23:16:28.973269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:35.701371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"79'216","reported_seq":1465,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.126248+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:16:26.126248+0000","last_peered":"2024-09-21T23:16:26.126248+0000","last_clean":"2024-09-21T23:16:26.126248+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:16:26.126248+0000","last_undegraded":"2024-09-21T23:16:26.126248+0000","last_fullsized":"2024-09-21T23:16:26.126248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:09.744658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":8675364,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":1181,"num_read_kb":7160,"num_write":212,"num_write_kb":3849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.021474+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:16:29.021474+0000","last_peered":"2024-09-21T23:16:29.021474+0000","last_clean":"2024-09-21T23:16:29.021474+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:16:29.021474+0000","last_undegraded":"2024-09-21T23:16:29.021474+0000","last_fullsized":"2024-09-21T23:16:29.021474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"81'235","reported_seq":7951,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.881346+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:16:29.881346+0000","last_peered":"2024-09-21T23:16:29.881346+0000","last_clean":"2024-09-21T23:16:29.881346+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:16:29.881346+0000","last_undegraded":"2024-09-21T23:16:29.881346+0000","last_fullsized":"2024-09-21T23:16:29.881346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:19:43.575782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":16361072,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":8474,"num_read_kb":20288,"num_write":231,"num_write_kb":2089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.022498+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:16:29.022498+0000","last_peered":"2024-09-21T23:16:29.022498+0000","last_clean":"2024-09-21T23:16:29.022498+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:16:29.022498+0000","last_undegraded":"2024-09-21T23:16:29.022498+0000","last_fullsized":"2024-09-21T23:16:29.022498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":238,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.962005+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:16:25.962005+0000","last_peered":"2024-09-21T23:16:25.962005+0000","last_clean":"2024-09-21T23:16:25.962005+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:16:25.962005+0000","last_undegraded":"2024-09-21T23:16:25.962005+0000","last_fullsized":"2024-09-21T23:16:25.962005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"81'529","reported_seq":10027,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.169009+0000","last_change":"2024-09-21T23:16:18.769531+0000","last_active":"2024-09-21T23:16:29.169009+0000","last_peered":"2024-09-21T23:16:29.169009+0000","last_clean":"2024-09-21T23:16:29.169009+0000","last_became_active":"2024-09-21T23:16:18.767995+0000","last_became_peered":"2024-09-21T23:16:18.767995+0000","last_unstale":"2024-09-21T23:16:29.169009+0000","last_undegraded":"2024-09-21T23:16:29.169009+0000","last_fullsized":"2024-09-21T23:16:29.169009+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:03:58.857828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":4778528,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":10594,"num_read_kb":14903,"num_write":315,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":230944,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"82'318","reported_seq":2936,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.816636+0000","last_change":"2024-09-21T23:15:03.325366+0000","last_active":"2024-09-21T23:16:29.816636+0000","last_peered":"2024-09-21T23:16:29.816636+0000","last_clean":"2024-09-21T23:16:29.816636+0000","last_became_active":"2024-09-21T23:15:01.329082+0000","last_became_peered":"2024-09-21T23:15:01.329082+0000","last_unstale":"2024-09-21T23:16:29.816636+0000","last_undegraded":"2024-09-21T23:16:29.816636+0000","last_fullsized":"2024-09-21T23:16:29.816636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:03:18.625255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":5206528,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":4314,"num_read_kb":7993,"num_write":229,"num_write_kb":877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"82'152","reported_seq":622,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.821102+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:16:29.821102+0000","last_peered":"2024-09-21T23:16:29.821102+0000","last_clean":"2024-09-21T23:16:29.821102+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:16:29.821102+0000","last_undegraded":"2024-09-21T23:16:29.821102+0000","last_fullsized":"2024-09-21T23:16:29.821102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:33.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685456,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":230,"num_read_kb":1025,"num_write":143,"num_write_kb":672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"82'345","reported_seq":4206,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.838034+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:16:29.838034+0000","last_peered":"2024-09-21T23:16:29.838034+0000","last_clean":"2024-09-21T23:16:29.838034+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:16:29.838034+0000","last_undegraded":"2024-09-21T23:16:29.838034+0000","last_fullsized":"2024-09-21T23:16:29.838034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:40.717873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":7649823,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":6187,"num_read_kb":14864,"num_write":239,"num_write_kb":992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"82'284","reported_seq":1412,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.816916+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:16:29.816916+0000","last_peered":"2024-09-21T23:16:29.816916+0000","last_clean":"2024-09-21T23:16:29.816916+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:16:29.816916+0000","last_undegraded":"2024-09-21T23:16:29.816916+0000","last_fullsized":"2024-09-21T23:16:29.816916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:23:13.590865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":13213229,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":1614,"num_read_kb":2785,"num_write":258,"num_write_kb":1277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"82'201","reported_seq":3336,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.721354+0000","last_change":"2024-09-21T23:15:06.357124+0000","last_active":"2024-09-21T23:16:29.721354+0000","last_peered":"2024-09-21T23:16:29.721354+0000","last_clean":"2024-09-21T23:16:29.721354+0000","last_became_active":"2024-09-21T23:15:06.356745+0000","last_became_peered":"2024-09-21T23:15:06.356745+0000","last_unstale":"2024-09-21T23:16:29.721354+0000","last_undegraded":"2024-09-21T23:16:29.721354+0000","last_fullsized":"2024-09-21T23:16:29.721354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:38.550466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5796620,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":3310,"num_read_kb":23574,"num_write":232,"num_write_kb":1172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"82'220","reported_seq":2108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.803733+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:16:29.803733+0000","last_peered":"2024-09-21T23:16:29.803733+0000","last_clean":"2024-09-21T23:16:29.803733+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:16:29.803733+0000","last_undegraded":"2024-09-21T23:16:29.803733+0000","last_fullsized":"2024-09-21T23:16:29.803733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:06:09.875028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":6611022,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":2915,"num_read_kb":8604,"num_write":227,"num_write_kb":804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'243","reported_seq":3166,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.023522+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:16:29.023522+0000","last_peered":"2024-09-21T23:16:29.023522+0000","last_clean":"2024-09-21T23:16:29.023522+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:16:29.023522+0000","last_undegraded":"2024-09-21T23:16:29.023522+0000","last_fullsized":"2024-09-21T23:16:29.023522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:01.157575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":5524496,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":4175,"num_read_kb":8334,"num_write":176,"num_write_kb":677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"82'210","reported_seq":1869,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.880721+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:16:29.880721+0000","last_peered":"2024-09-21T23:16:29.880721+0000","last_clean":"2024-09-21T23:16:29.880721+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:16:29.880721+0000","last_undegraded":"2024-09-21T23:16:29.880721+0000","last_fullsized":"2024-09-21T23:16:29.880721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:27:18.442675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":10912811,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1448,"num_read_kb":9466,"num_write":194,"num_write_kb":1176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"82'348","reported_seq":1947,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.035740+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:16:29.035740+0000","last_peered":"2024-09-21T23:16:29.035740+0000","last_clean":"2024-09-21T23:16:29.035740+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:16:29.035740+0000","last_undegraded":"2024-09-21T23:16:29.035740+0000","last_fullsized":"2024-09-21T23:16:29.035740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:53:08.993392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3520037,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":2547,"num_read_kb":5982,"num_write":508,"num_write_kb":1130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"79'362","reported_seq":16078,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.107101+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:16:26.107101+0000","last_peered":"2024-09-21T23:16:26.107101+0000","last_clean":"2024-09-21T23:16:26.107101+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:16:26.107101+0000","last_undegraded":"2024-09-21T23:16:26.107101+0000","last_fullsized":"2024-09-21T23:16:26.107101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:56.852354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":23640636,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":17757,"num_read_kb":29245,"num_write":256,"num_write_kb":1924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"82'172","reported_seq":7719,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.022626+0000","last_change":"2024-09-21T23:15:06.345498+0000","last_active":"2024-09-21T23:16:29.022626+0000","last_peered":"2024-09-21T23:16:29.022626+0000","last_clean":"2024-09-21T23:16:29.022626+0000","last_became_active":"2024-09-21T23:15:06.344491+0000","last_became_peered":"2024-09-21T23:15:06.344491+0000","last_unstale":"2024-09-21T23:16:29.022626+0000","last_undegraded":"2024-09-21T23:16:29.022626+0000","last_fullsized":"2024-09-21T23:16:29.022626+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:56.602516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7459344,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":7441,"num_read_kb":16415,"num_write":158,"num_write_kb":767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":335488463,"num_objects":759,"num_object_clones":174,"num_object_copies":1518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":759,"num_whiteouts":348,"num_read":161407,"num_read_kb":422732,"num_write":10693,"num_write_kb":64961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1403,"num_bytes_recovered":15900820,"num_keys_recovered":199,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":47149056,"data_stored":57148994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":9499,"ondisk_log_size":9499,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607577,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50424,"kb_used_data":10908,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321416,"statfs":{"total":96636764160,"available":96585129984,"internally_reserved":0,"allocated":11169792,"data_stored":12991544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607577,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38420,"kb_used_data":11388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333420,"statfs":{"total":96636764160,"available":96597422080,"internally_reserved":0,"allocated":11661312,"data_stored":14545165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607577,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607577,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60896,"kb_used_data":7500,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310944,"statfs":{"total":96636764160,"available":96574406656,"internally_reserved":0,"allocated":7680000,"data_stored":8796516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607578,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64588,"kb_used_data":11192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307252,"statfs":{"total":96636764160,"available":96570626048,"internally_reserved":0,"allocated":11460608,"data_stored":11931353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005062,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":9068,"kb_used_omap":5,"kb_used_meta":32314,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":9285632,"data_stored":10222518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":33089829},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8433664,"data_stored":9899589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10473472,"data_stored":11504860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7008256,"data_stored":8684429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10731520,"data_stored":14176566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10498048,"data_stored":12883534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:16:30.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:16:30.503 INFO:teuthology.orchestra.run.smithi170.stdout:2985 read 0x1b91800 thru 0x1b926c2 (0xec3 bytes) 2024-09-21T23:16:30.525 INFO:teuthology.orchestra.run.smithi170.stdout:2986 write 0x4c5d400 thru 0x4c6909b (0xbc9c bytes) 2024-09-21T23:16:30.527 INFO:teuthology.orchestra.run.smithi170.stdout:2988 read 0xd75200 thru 0xd81190 (0xbf91 bytes) 2024-09-21T23:16:30.531 INFO:teuthology.orchestra.run.smithi170.stdout:2989 read 0x3852e00 thru 0x385f812 (0xca13 bytes) 2024-09-21T23:16:30.537 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:30.532+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:30.539 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:30.539 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:30.553 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:30.595 INFO:teuthology.orchestra.run.smithi170.stdout:2990 write 0xe6fae00 thru 0xe704789 (0x998a bytes) 2024-09-21T23:16:30.597 INFO:teuthology.orchestra.run.smithi170.stdout:2992 read 0x722b600 thru 0x7236fb7 (0xb9b8 bytes) 2024-09-21T23:16:30.598 INFO:teuthology.orchestra.run.smithi170.stdout:2994 write 0x7bec800 thru 0x7bf63c4 (0x9bc5 bytes) 2024-09-21T23:16:30.598 INFO:teuthology.orchestra.run.smithi170.stdout:2995 write 0x4910e00 thru 0x49110ee (0x2ef bytes) 2024-09-21T23:16:30.599 INFO:teuthology.orchestra.run.smithi170.stdout:2996 write 0xe5c7e00 thru 0xe5ce092 (0x6293 bytes) 2024-09-21T23:16:30.599 INFO:teuthology.orchestra.run.smithi170.stdout:2997 read 0xe1dfc00 thru 0xe1e7069 (0x746a bytes) 2024-09-21T23:16:30.600 INFO:teuthology.orchestra.run.smithi170.stdout:2999 trunc from 0xe704800 to 0xe84b800 2024-09-21T23:16:30.642 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:30.640+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:30.650 INFO:teuthology.orchestra.run.smithi170.stdout:3000 write 0x724b400 thru 0x724dbbb (0x27bc bytes) 2024-09-21T23:16:30.670 INFO:teuthology.orchestra.run.smithi170.stdout:3002 trunc from 0xe84b800 to 0x8335600 2024-09-21T23:16:30.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:30.713 INFO:teuthology.orchestra.run.smithi170.stdout:3004 write 0x1ef7800 thru 0x1f01c05 (0xa406 bytes) 2024-09-21T23:16:30.728 INFO:teuthology.orchestra.run.smithi170.stdout:3006 trunc from 0x8335600 to 0x290a200 2024-09-21T23:16:30.728 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:30.728 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:30.738 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:30.736+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:30.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:30.788 INFO:teuthology.orchestra.run.smithi170.stdout:3007 punch from 0x630000 to 0x632c00, (0x2c00 bytes) 2024-09-21T23:16:30.793 INFO:teuthology.orchestra.run.smithi170.stdout:3008 punch from 0x408a00 to 0x410800, (0x7e00 bytes) 2024-09-21T23:16:30.811 INFO:teuthology.orchestra.run.smithi170.stdout:3009 write 0x10f000 thru 0x115d61 (0x6d62 bytes) 2024-09-21T23:16:30.828 INFO:teuthology.orchestra.run.smithi170.stdout:3010 read 0x2829800 thru 0x2837a3d (0xe23e bytes) 2024-09-21T23:16:30.829 INFO:teuthology.orchestra.run.smithi170.stdout:3014 read 0x1482e00 thru 0x1485de1 (0x2fe2 bytes) 2024-09-21T23:16:30.839 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:30.836+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:3016 read 0x1913400 thru 0x1921717 (0xe318 bytes) 2024-09-21T23:16:30.875 INFO:teuthology.orchestra.run.smithi170.stdout:3017 write 0x1e52800 thru 0x1e5cac2 (0xa2c3 bytes) 2024-09-21T23:16:30.893 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:30.893 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:30.918 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:30.918 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:30.921 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:30.921 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:30.927 INFO:teuthology.orchestra.run.smithi170.stdout:3019 write 0x8f5b400 thru 0x8f6172b (0x632c bytes) 2024-09-21T23:16:30.929 INFO:teuthology.orchestra.run.smithi170.stdout:3022 write 0x745e00 thru 0x74c588 (0x6789 bytes) 2024-09-21T23:16:30.930 INFO:teuthology.orchestra.run.smithi170.stdout:3025 read 0x537a400 thru 0x537d050 (0x2c51 bytes) 2024-09-21T23:16:30.932 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:30.939 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:30.939 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:30.940 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:30.940+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:30.942 INFO:teuthology.orchestra.run.smithi170.stdout:3026 trunc from 0x8f61800 to 0x46baa00 2024-09-21T23:16:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:30.949 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:16:30.969 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:30.969 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:30.979 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-21T23:16:30.871382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336078850,"num_objects":763,"num_object_clones":174,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":348,"num_read":161523,"num_read_kb":422832,"num_write":10854,"num_write_kb":67379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1409,"num_bytes_recovered":17081594,"num_keys_recovered":199,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9619,"ondisk_log_size":9619,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":255736,"kb_used_data":50056,"kb_used_omap":11,"kb_used_meta":205556,"kb_avail":471603464,"statfs":{"total":483183820800,"available":482921947136,"internally_reserved":0,"allocated":51257344,"data_stored":58487096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12231,"internal_metadata":210489401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19179019,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":5,"num_read":8862,"num_read_kb":17060,"num_write":426,"num_write_kb":1940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000889"},"pg_stats":[{"pgid":"3.1f","version":"79'268","reported_seq":9065,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.113749+0000","last_change":"2024-09-21T23:16:06.209669+0000","last_active":"2024-09-21T23:16:26.113749+0000","last_peered":"2024-09-21T23:16:26.113749+0000","last_clean":"2024-09-21T23:16:26.113749+0000","last_became_active":"2024-09-21T23:16:06.208917+0000","last_became_peered":"2024-09-21T23:16:06.208917+0000","last_unstale":"2024-09-21T23:16:26.113749+0000","last_undegraded":"2024-09-21T23:16:26.113749+0000","last_fullsized":"2024-09-21T23:16:26.113749+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:11:21.474520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":10777177,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":9327,"num_read_kb":9167,"num_write":356,"num_write_kb":822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":14388,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"82'202","reported_seq":1207,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.826524+0000","last_change":"2024-09-21T23:16:18.771067+0000","last_active":"2024-09-21T23:16:29.826524+0000","last_peered":"2024-09-21T23:16:29.826524+0000","last_clean":"2024-09-21T23:16:29.826524+0000","last_became_active":"2024-09-21T23:16:18.768784+0000","last_became_peered":"2024-09-21T23:16:18.768784+0000","last_unstale":"2024-09-21T23:16:29.826524+0000","last_undegraded":"2024-09-21T23:16:29.826524+0000","last_fullsized":"2024-09-21T23:16:29.826524+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:51:53.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":9015860,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":738,"num_read_kb":9336,"num_write":189,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2209332,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"82'102","reported_seq":1366,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.719645+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:16:29.719645+0000","last_peered":"2024-09-21T23:16:29.719645+0000","last_clean":"2024-09-21T23:16:29.719645+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:16:29.719645+0000","last_undegraded":"2024-09-21T23:16:29.719645+0000","last_fullsized":"2024-09-21T23:16:29.719645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:51:33.292362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":4909568,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":1143,"num_read_kb":6797,"num_write":97,"num_write_kb":2013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"81'267","reported_seq":3417,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.881351+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:16:29.881351+0000","last_peered":"2024-09-21T23:16:29.881351+0000","last_clean":"2024-09-21T23:16:29.881351+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:16:29.881351+0000","last_undegraded":"2024-09-21T23:16:29.881351+0000","last_fullsized":"2024-09-21T23:16:29.881351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:18:44.303903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":16168528,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":3835,"num_read_kb":17775,"num_write":239,"num_write_kb":10571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"68'244","reported_seq":2081,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.961873+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:16:25.961873+0000","last_peered":"2024-09-21T23:16:25.961873+0000","last_clean":"2024-09-21T23:16:25.961873+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:16:25.961873+0000","last_undegraded":"2024-09-21T23:16:25.961873+0000","last_fullsized":"2024-09-21T23:16:25.961873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:43:24.435448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":5896734,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":2700,"num_read_kb":11597,"num_write":259,"num_write_kb":876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"82'302","reported_seq":5694,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.740041+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:16:29.740041+0000","last_peered":"2024-09-21T23:16:29.740041+0000","last_clean":"2024-09-21T23:16:29.740041+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:16:29.740041+0000","last_undegraded":"2024-09-21T23:16:29.740041+0000","last_fullsized":"2024-09-21T23:16:29.740041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:04:38.638877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":6019619,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":6947,"num_read_kb":16872,"num_write":222,"num_write_kb":794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"82'142","reported_seq":1278,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.800520+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:16:29.800520+0000","last_peered":"2024-09-21T23:16:29.800520+0000","last_clean":"2024-09-21T23:16:29.800520+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:16:29.800520+0000","last_undegraded":"2024-09-21T23:16:29.800520+0000","last_fullsized":"2024-09-21T23:16:29.800520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:28:11.479193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":8011792,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":969,"num_read_kb":6565,"num_write":132,"num_write_kb":1015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"82'376","reported_seq":10673,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.820617+0000","last_change":"2024-09-21T23:16:06.209284+0000","last_active":"2024-09-21T23:16:29.820617+0000","last_peered":"2024-09-21T23:16:29.820617+0000","last_clean":"2024-09-21T23:16:29.820617+0000","last_became_active":"2024-09-21T23:16:06.208604+0000","last_became_peered":"2024-09-21T23:16:06.208604+0000","last_unstale":"2024-09-21T23:16:29.820617+0000","last_undegraded":"2024-09-21T23:16:29.820617+0000","last_fullsized":"2024-09-21T23:16:29.820617+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:45.657512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":24008252,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":11223,"num_read_kb":24431,"num_write":523,"num_write_kb":8284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":75324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"68'170","reported_seq":8903,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.961964+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:16:25.961964+0000","last_peered":"2024-09-21T23:16:25.961964+0000","last_clean":"2024-09-21T23:16:25.961964+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:16:25.961964+0000","last_undegraded":"2024-09-21T23:16:25.961964+0000","last_fullsized":"2024-09-21T23:16:25.961964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:58.546596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":11476032,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":8592,"num_read_kb":15658,"num_write":157,"num_write_kb":683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"82'351","reported_seq":2059,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.881329+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:16:29.881329+0000","last_peered":"2024-09-21T23:16:29.881329+0000","last_clean":"2024-09-21T23:16:29.881329+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:16:29.881329+0000","last_undegraded":"2024-09-21T23:16:29.881329+0000","last_fullsized":"2024-09-21T23:16:29.881329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:09:58.769914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":31935138,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":30,"num_read":2158,"num_read_kb":7024,"num_write":438,"num_write_kb":2627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"69'122","reported_seq":2954,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.962119+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:16:25.962119+0000","last_peered":"2024-09-21T23:16:25.962119+0000","last_clean":"2024-09-21T23:16:25.962119+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:16:25.962119+0000","last_undegraded":"2024-09-21T23:16:25.962119+0000","last_fullsized":"2024-09-21T23:16:25.962119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:11:29.668035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898304,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":2713,"num_read_kb":24295,"num_write":120,"num_write_kb":2627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"82'497","reported_seq":1858,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.805335+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:16:29.805335+0000","last_peered":"2024-09-21T23:16:29.805335+0000","last_clean":"2024-09-21T23:16:29.805335+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:16:29.805335+0000","last_undegraded":"2024-09-21T23:16:29.805335+0000","last_fullsized":"2024-09-21T23:16:29.805335+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:42.653598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8018079,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":3022,"num_read_kb":11682,"num_write":736,"num_write_kb":1407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.036285+0000","last_change":"2024-09-21T23:15:06.350164+0000","last_active":"2024-09-21T23:16:29.036285+0000","last_peered":"2024-09-21T23:16:29.036285+0000","last_clean":"2024-09-21T23:16:29.036285+0000","last_became_active":"2024-09-21T23:15:06.349271+0000","last_became_peered":"2024-09-21T23:15:06.349271+0000","last_unstale":"2024-09-21T23:16:29.036285+0000","last_undegraded":"2024-09-21T23:16:29.036285+0000","last_fullsized":"2024-09-21T23:16:29.036285+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:56.638782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'174","reported_seq":17156,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.048887+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:16:26.048887+0000","last_peered":"2024-09-21T23:16:26.048887+0000","last_clean":"2024-09-21T23:16:26.048887+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:16:26.048887+0000","last_undegraded":"2024-09-21T23:16:26.048887+0000","last_fullsized":"2024-09-21T23:16:26.048887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:59:22.879499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":25137192,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":16534,"num_read_kb":24043,"num_write":164,"num_write_kb":2086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.021456+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:16:29.021456+0000","last_peered":"2024-09-21T23:16:29.021456+0000","last_clean":"2024-09-21T23:16:29.021456+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:16:29.021456+0000","last_undegraded":"2024-09-21T23:16:29.021456+0000","last_fullsized":"2024-09-21T23:16:29.021456+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"82'1475","reported_seq":4360,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:28.974011+0000","last_change":"2024-09-21T23:16:18.776075+0000","last_active":"2024-09-21T23:16:28.974011+0000","last_peered":"2024-09-21T23:16:28.974011+0000","last_clean":"2024-09-21T23:16:28.974011+0000","last_became_active":"2024-09-21T23:16:18.771731+0000","last_became_peered":"2024-09-21T23:16:18.771731+0000","last_unstale":"2024-09-21T23:16:28.974011+0000","last_undegraded":"2024-09-21T23:16:28.974011+0000","last_fullsized":"2024-09-21T23:16:28.974011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1475,"log_dups_size":0,"ondisk_log_size":1475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:42:09.391659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":8584106,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":9811,"num_read_kb":29148,"num_write":2652,"num_write_kb":5385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":2243370,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.961969+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:16:25.961969+0000","last_peered":"2024-09-21T23:16:25.961969+0000","last_clean":"2024-09-21T23:16:25.961969+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:16:25.961969+0000","last_undegraded":"2024-09-21T23:16:25.961969+0000","last_fullsized":"2024-09-21T23:16:25.961969+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"82'257","reported_seq":997,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.820494+0000","last_change":"2024-09-21T23:16:17.766149+0000","last_active":"2024-09-21T23:16:29.820494+0000","last_peered":"2024-09-21T23:16:29.820494+0000","last_clean":"2024-09-21T23:16:29.820494+0000","last_became_active":"2024-09-21T23:16:17.765404+0000","last_became_peered":"2024-09-21T23:16:17.765404+0000","last_unstale":"2024-09-21T23:16:29.820494+0000","last_undegraded":"2024-09-21T23:16:29.820494+0000","last_fullsized":"2024-09-21T23:16:29.820494+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:50:03.304250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":5189155,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1609,"num_read_kb":2627,"num_write":305,"num_write_kb":765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":113664,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.021291+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:16:29.021291+0000","last_peered":"2024-09-21T23:16:29.021291+0000","last_clean":"2024-09-21T23:16:29.021291+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:16:29.021291+0000","last_undegraded":"2024-09-21T23:16:29.021291+0000","last_fullsized":"2024-09-21T23:16:29.021291+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"82'294","reported_seq":2455,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.796392+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:16:29.796392+0000","last_peered":"2024-09-21T23:16:29.796392+0000","last_clean":"2024-09-21T23:16:29.796392+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:16:29.796392+0000","last_undegraded":"2024-09-21T23:16:29.796392+0000","last_fullsized":"2024-09-21T23:16:29.796392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:02:11.655743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":10286407,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3391,"num_read_kb":14958,"num_write":338,"num_write_kb":1526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'208","reported_seq":2354,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.802934+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:16:29.802934+0000","last_peered":"2024-09-21T23:16:29.802934+0000","last_clean":"2024-09-21T23:16:29.802934+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:16:29.802934+0000","last_undegraded":"2024-09-21T23:16:29.802934+0000","last_fullsized":"2024-09-21T23:16:29.802934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:22:27.188859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":11370014,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":2056,"num_read_kb":12156,"num_write":209,"num_write_kb":853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.022547+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:16:29.022547+0000","last_peered":"2024-09-21T23:16:29.022547+0000","last_clean":"2024-09-21T23:16:29.022547+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:16:29.022547+0000","last_undegraded":"2024-09-21T23:16:29.022547+0000","last_fullsized":"2024-09-21T23:16:29.022547+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"82'213","reported_seq":2786,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.824090+0000","last_change":"2024-09-21T23:16:18.772875+0000","last_active":"2024-09-21T23:16:29.824090+0000","last_peered":"2024-09-21T23:16:29.824090+0000","last_clean":"2024-09-21T23:16:29.824090+0000","last_became_active":"2024-09-21T23:16:18.770556+0000","last_became_peered":"2024-09-21T23:16:18.770556+0000","last_unstale":"2024-09-21T23:16:29.824090+0000","last_undegraded":"2024-09-21T23:16:29.824090+0000","last_fullsized":"2024-09-21T23:16:29.824090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:12:10.034857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":3751540,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":2452,"num_read_kb":7963,"num_write":179,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":80516,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:28.973269+0000","last_change":"2024-09-21T23:16:15.257881+0000","last_active":"2024-09-21T23:16:28.973269+0000","last_peered":"2024-09-21T23:16:28.973269+0000","last_clean":"2024-09-21T23:16:28.973269+0000","last_became_active":"2024-09-21T23:16:15.257496+0000","last_became_peered":"2024-09-21T23:16:15.257496+0000","last_unstale":"2024-09-21T23:16:28.973269+0000","last_undegraded":"2024-09-21T23:16:28.973269+0000","last_fullsized":"2024-09-21T23:16:28.973269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:35.701371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"79'216","reported_seq":1465,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.126248+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:16:26.126248+0000","last_peered":"2024-09-21T23:16:26.126248+0000","last_clean":"2024-09-21T23:16:26.126248+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:16:26.126248+0000","last_undegraded":"2024-09-21T23:16:26.126248+0000","last_fullsized":"2024-09-21T23:16:26.126248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:09.744658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":8675364,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":1181,"num_read_kb":7160,"num_write":212,"num_write_kb":3849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.021474+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:16:29.021474+0000","last_peered":"2024-09-21T23:16:29.021474+0000","last_clean":"2024-09-21T23:16:29.021474+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:16:29.021474+0000","last_undegraded":"2024-09-21T23:16:29.021474+0000","last_fullsized":"2024-09-21T23:16:29.021474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"81'235","reported_seq":7951,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.881346+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:16:29.881346+0000","last_peered":"2024-09-21T23:16:29.881346+0000","last_clean":"2024-09-21T23:16:29.881346+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:16:29.881346+0000","last_undegraded":"2024-09-21T23:16:29.881346+0000","last_fullsized":"2024-09-21T23:16:29.881346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:19:43.575782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":16361072,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":8474,"num_read_kb":20288,"num_write":231,"num_write_kb":2089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.022498+0000","last_change":"2024-09-21T23:14:53.266479+0000","last_active":"2024-09-21T23:16:29.022498+0000","last_peered":"2024-09-21T23:16:29.022498+0000","last_clean":"2024-09-21T23:16:29.022498+0000","last_became_active":"2024-09-21T23:14:51.255564+0000","last_became_peered":"2024-09-21T23:14:51.255564+0000","last_unstale":"2024-09-21T23:16:29.022498+0000","last_undegraded":"2024-09-21T23:16:29.022498+0000","last_fullsized":"2024-09-21T23:16:29.022498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:43:20.843933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":238,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:25.962005+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:16:25.962005+0000","last_peered":"2024-09-21T23:16:25.962005+0000","last_clean":"2024-09-21T23:16:25.962005+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:16:25.962005+0000","last_undegraded":"2024-09-21T23:16:25.962005+0000","last_fullsized":"2024-09-21T23:16:25.962005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"81'529","reported_seq":10027,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.169009+0000","last_change":"2024-09-21T23:16:18.769531+0000","last_active":"2024-09-21T23:16:29.169009+0000","last_peered":"2024-09-21T23:16:29.169009+0000","last_clean":"2024-09-21T23:16:29.169009+0000","last_became_active":"2024-09-21T23:16:18.767995+0000","last_became_peered":"2024-09-21T23:16:18.767995+0000","last_unstale":"2024-09-21T23:16:29.169009+0000","last_undegraded":"2024-09-21T23:16:29.169009+0000","last_fullsized":"2024-09-21T23:16:29.169009+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:03:58.857828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":4778528,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":10594,"num_read_kb":14903,"num_write":315,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":230944,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"82'318","reported_seq":2936,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.816636+0000","last_change":"2024-09-21T23:15:03.325366+0000","last_active":"2024-09-21T23:16:29.816636+0000","last_peered":"2024-09-21T23:16:29.816636+0000","last_clean":"2024-09-21T23:16:29.816636+0000","last_became_active":"2024-09-21T23:15:01.329082+0000","last_became_peered":"2024-09-21T23:15:01.329082+0000","last_unstale":"2024-09-21T23:16:29.816636+0000","last_undegraded":"2024-09-21T23:16:29.816636+0000","last_fullsized":"2024-09-21T23:16:29.816636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:03:18.625255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":5206528,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":4314,"num_read_kb":7993,"num_write":229,"num_write_kb":877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"82'152","reported_seq":622,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.821102+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:16:29.821102+0000","last_peered":"2024-09-21T23:16:29.821102+0000","last_clean":"2024-09-21T23:16:29.821102+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:16:29.821102+0000","last_undegraded":"2024-09-21T23:16:29.821102+0000","last_fullsized":"2024-09-21T23:16:29.821102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:33.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685456,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":230,"num_read_kb":1025,"num_write":143,"num_write_kb":672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"82'345","reported_seq":4206,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.838034+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:16:29.838034+0000","last_peered":"2024-09-21T23:16:29.838034+0000","last_clean":"2024-09-21T23:16:29.838034+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:16:29.838034+0000","last_undegraded":"2024-09-21T23:16:29.838034+0000","last_fullsized":"2024-09-21T23:16:29.838034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:40.717873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":7649823,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":6187,"num_read_kb":14864,"num_write":239,"num_write_kb":992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"82'284","reported_seq":1412,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.816916+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:16:29.816916+0000","last_peered":"2024-09-21T23:16:29.816916+0000","last_clean":"2024-09-21T23:16:29.816916+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:16:29.816916+0000","last_undegraded":"2024-09-21T23:16:29.816916+0000","last_fullsized":"2024-09-21T23:16:29.816916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:23:13.590865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":13213229,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":1614,"num_read_kb":2785,"num_write":258,"num_write_kb":1277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"82'201","reported_seq":3336,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.721354+0000","last_change":"2024-09-21T23:15:06.357124+0000","last_active":"2024-09-21T23:16:29.721354+0000","last_peered":"2024-09-21T23:16:29.721354+0000","last_clean":"2024-09-21T23:16:29.721354+0000","last_became_active":"2024-09-21T23:15:06.356745+0000","last_became_peered":"2024-09-21T23:15:06.356745+0000","last_unstale":"2024-09-21T23:16:29.721354+0000","last_undegraded":"2024-09-21T23:16:29.721354+0000","last_fullsized":"2024-09-21T23:16:29.721354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:07:38.550466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5796620,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":3310,"num_read_kb":23574,"num_write":232,"num_write_kb":1172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"82'220","reported_seq":2108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.803733+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:16:29.803733+0000","last_peered":"2024-09-21T23:16:29.803733+0000","last_clean":"2024-09-21T23:16:29.803733+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:16:29.803733+0000","last_undegraded":"2024-09-21T23:16:29.803733+0000","last_fullsized":"2024-09-21T23:16:29.803733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:06:09.875028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":6611022,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":2915,"num_read_kb":8604,"num_write":227,"num_write_kb":804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"82'243","reported_seq":3166,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.023522+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:16:29.023522+0000","last_peered":"2024-09-21T23:16:29.023522+0000","last_clean":"2024-09-21T23:16:29.023522+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:16:29.023522+0000","last_undegraded":"2024-09-21T23:16:29.023522+0000","last_fullsized":"2024-09-21T23:16:29.023522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:12:01.157575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":5524496,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":4175,"num_read_kb":8334,"num_write":176,"num_write_kb":677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"82'210","reported_seq":1869,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.880721+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:16:29.880721+0000","last_peered":"2024-09-21T23:16:29.880721+0000","last_clean":"2024-09-21T23:16:29.880721+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:16:29.880721+0000","last_undegraded":"2024-09-21T23:16:29.880721+0000","last_fullsized":"2024-09-21T23:16:29.880721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:27:18.442675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":10912811,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":1448,"num_read_kb":9466,"num_write":194,"num_write_kb":1176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"82'348","reported_seq":1947,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.035740+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:16:29.035740+0000","last_peered":"2024-09-21T23:16:29.035740+0000","last_clean":"2024-09-21T23:16:29.035740+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:16:29.035740+0000","last_undegraded":"2024-09-21T23:16:29.035740+0000","last_fullsized":"2024-09-21T23:16:29.035740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:53:08.993392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3520037,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":2547,"num_read_kb":5982,"num_write":508,"num_write_kb":1130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"79'362","reported_seq":16078,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T23:16:26.107101+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:16:26.107101+0000","last_peered":"2024-09-21T23:16:26.107101+0000","last_clean":"2024-09-21T23:16:26.107101+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:16:26.107101+0000","last_undegraded":"2024-09-21T23:16:26.107101+0000","last_fullsized":"2024-09-21T23:16:26.107101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:56.852354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":23640636,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":17757,"num_read_kb":29245,"num_write":256,"num_write_kb":1924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"82'172","reported_seq":7719,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T23:16:29.022626+0000","last_change":"2024-09-21T23:15:06.345498+0000","last_active":"2024-09-21T23:16:29.022626+0000","last_peered":"2024-09-21T23:16:29.022626+0000","last_clean":"2024-09-21T23:16:29.022626+0000","last_became_active":"2024-09-21T23:15:06.344491+0000","last_became_peered":"2024-09-21T23:15:06.344491+0000","last_unstale":"2024-09-21T23:16:29.022626+0000","last_undegraded":"2024-09-21T23:16:29.022626+0000","last_fullsized":"2024-09-21T23:16:29.022626+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:56.602516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7459344,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":7441,"num_read_kb":16415,"num_write":158,"num_write_kb":767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":335488463,"num_objects":759,"num_object_clones":174,"num_object_copies":1518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":759,"num_whiteouts":348,"num_read":161407,"num_read_kb":422732,"num_write":10693,"num_write_kb":64961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1403,"num_bytes_recovered":15900820,"num_keys_recovered":199,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":47149056,"data_stored":57148994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":9499,"ondisk_log_size":9499,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607577,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50424,"kb_used_data":10908,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321416,"statfs":{"total":96636764160,"available":96585129984,"internally_reserved":0,"allocated":11169792,"data_stored":12991544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607577,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38420,"kb_used_data":11388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333420,"statfs":{"total":96636764160,"available":96597422080,"internally_reserved":0,"allocated":11661312,"data_stored":14545165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607577,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607577,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60896,"kb_used_data":7500,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310944,"statfs":{"total":96636764160,"available":96574406656,"internally_reserved":0,"allocated":7680000,"data_stored":8796516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607578,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64588,"kb_used_data":11192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307252,"statfs":{"total":96636764160,"available":96570626048,"internally_reserved":0,"allocated":11460608,"data_stored":11931353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005062,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":9068,"kb_used_omap":5,"kb_used_meta":32314,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":9285632,"data_stored":10222518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":33089829},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8433664,"data_stored":9899589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10473472,"data_stored":11504860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7008256,"data_stored":8684429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10731520,"data_stored":14176566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10498048,"data_stored":12883534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:16:30.981 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:16:30.981 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:16:30.981 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.125067768478616 2024-09-21T23:16:30.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.125067768478616 2024-09-21T23:16:30.994 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:30.994 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:31.017 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:31.028 INFO:teuthology.orchestra.run.smithi170.stdout:3027 write 0x61ad600 thru 0x61b37ed (0x61ee bytes) 2024-09-21T23:16:31.030 INFO:teuthology.orchestra.run.smithi170.stdout:3030 read 0x1ac5e00 thru 0x1ac7315 (0x1516 bytes) 2024-09-21T23:16:31.032 INFO:teuthology.orchestra.run.smithi170.stdout:3031 read 0x48d8200 thru 0x48e8177 (0xff78 bytes) 2024-09-21T23:16:31.032 INFO:teuthology.orchestra.run.smithi170.stdout:3032 read 0x1661000 thru 0x166db95 (0xcb96 bytes) 2024-09-21T23:16:31.035 INFO:teuthology.orchestra.run.smithi170.stdout:3033 trunc from 0x61b3800 to 0x6b2a800 2024-09-21T23:16:31.055 INFO:teuthology.orchestra.run.smithi170.stdout:3035 punch from 0x3a16600 to 0x3a22a00, (0xc400 bytes) 2024-09-21T23:16:31.057 INFO:teuthology.orchestra.run.smithi170.stdout:3036 write 0x6392c00 thru 0x63952f0 (0x26f1 bytes) 2024-09-21T23:16:31.057 INFO:teuthology.orchestra.run.smithi170.stdout:3037 read 0xaad200 thru 0xab0811 (0x3612 bytes) 2024-09-21T23:16:31.068 INFO:teuthology.orchestra.run.smithi170.stdout:3038 write 0x607fc00 thru 0x60883cc (0x87cd bytes) 2024-09-21T23:16:31.072 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:31.072 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:31.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.072 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5313.0:11944 3.13 3:cc5ffdab:::rbd_data.10f1c05601e9.0000000000000006:head [read 188416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-21T23:16:31.072 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:31.071308+0000", 2024-09-21T23:16:31.072 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.000110976, 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015322600000000001, 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5313", 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/243242205", 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 11944 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071308+0000", 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071308+0000", 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071309+0000", 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1200000000000002e-07 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071313+0000", 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9940000000000002e-06 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071323+0000", 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0217999999999999e-05 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071328+0000", 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1800000000000004e-06 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071387+0000", 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8527999999999999e-05 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:31.071431+0000", 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4793999999999998e-05 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:16:31.076 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:16:31.077 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:31.077 INFO:teuthology.orchestra.run.smithi170.stdout:3039 trunc from 0x6b2a800 to 0x83f6c00 2024-09-21T23:16:31.079 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:31.080+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:31.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:31.112 INFO:teuthology.orchestra.run.smithi170.stdout:3043 write 0x9a71800 thru 0x9a7594b (0x414c bytes) 2024-09-21T23:16:31.114 INFO:teuthology.orchestra.run.smithi170.stdout:3044 punch from 0x39eb400 to 0x39f4c00, (0x9800 bytes) 2024-09-21T23:16:31.115 INFO:teuthology.orchestra.run.smithi170.stdout:3045 read 0x16a9200 thru 0x16b303a (0x9e3b bytes) 2024-09-21T23:16:31.118 INFO:teuthology.orchestra.run.smithi170.stdout:3048 punch from 0x442a200 to 0x4430c00, (0x6a00 bytes) 2024-09-21T23:16:31.119 INFO:teuthology.orchestra.run.smithi170.stdout:3053 write 0x8c55c00 thru 0x8c5aeff (0x5300 bytes) 2024-09-21T23:16:31.119 INFO:teuthology.orchestra.run.smithi170.stdout:3055 read 0x227da00 thru 0x228b23d (0xd83e bytes) 2024-09-21T23:16:31.120 INFO:teuthology.orchestra.run.smithi170.stdout:3056 punch from 0x6144e00 to 0x6152c00, (0xde00 bytes) 2024-09-21T23:16:31.132 INFO:teuthology.orchestra.run.smithi170.stdout:3060 trunc from 0x9a75a00 to 0xedfd400 2024-09-21T23:16:31.142 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:31.140+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:31.152 INFO:teuthology.orchestra.run.smithi170.stdout:3062 write 0xdb42400 thru 0xdb4abd7 (0x87d8 bytes) 2024-09-21T23:16:31.153 INFO:teuthology.orchestra.run.smithi170.stdout:3065 write 0xe6fa800 thru 0xe703ac3 (0x92c4 bytes) 2024-09-21T23:16:31.154 INFO:teuthology.orchestra.run.smithi170.stdout:3066 write 0x726a200 thru 0x727125b (0x705c bytes) 2024-09-21T23:16:31.154 INFO:teuthology.orchestra.run.smithi170.stdout:3067 read 0xcc71200 thru 0xcc76252 (0x5053 bytes) 2024-09-21T23:16:31.155 INFO:teuthology.orchestra.run.smithi170.stdout:3070 trunc from 0xedfd400 to 0xe971200 2024-09-21T23:16:31.200 INFO:teuthology.orchestra.run.smithi170.stdout:3072 write 0x177e400 thru 0x1789463 (0xb064 bytes) 2024-09-21T23:16:31.202 INFO:teuthology.orchestra.run.smithi170.stdout:3074 clone 19 order 21 su 65536 sc 15 2024-09-21T23:16:31.224 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:31.224 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:31.239 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:31.243 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:31.244+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:31.344 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:31.344+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.276647721, 2024-09-21T23:16:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:16:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10124 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6156288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272290083000001, 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038330863999999999, 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10124 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0399999999999995e-07 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125818+0000", 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:16:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125827+0000", 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6330000000000002e-06 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125834+0000", 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9199999999999998e-06 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.163896+0000", 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038062072000000002 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164009+0000", 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011386400000000001 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164145+0000", 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013601100000000001 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10125 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6141952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272251712999999, 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038589275999999999, 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10125 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125854+0000", 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125856+0000", 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8159999999999998e-06 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125865+0000", 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7739999999999993e-06 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125872+0000", 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164189+0000", 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038317276999999997 2024-09-21T23:16:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164304+0000", 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114676 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164442+0000", 2024-09-21T23:16:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013814 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10126 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6172672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272223929999999, 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038865093000000003, 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10126 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125884+0000", 2024-09-21T23:16:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125893+0000", 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125900+0000", 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0500000000000003e-06 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164491+0000", 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038591473000000001 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164606+0000", 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115109 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164746+0000", 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013963800000000001 2024-09-21T23:16:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272196564000001, 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:16:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:16:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272169090999999, 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:16:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:16:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:16:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:16:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:16:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:16:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:16:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:16:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272128918, 2024-09-21T23:16:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:16:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:16:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:16:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272078148999999, 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:16:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272047566000001, 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:16:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.272019660000002, 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271992439999998, 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:16:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:16:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271965520999998, 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:16:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:16:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:16:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:16:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271909758, 2024-09-21T23:16:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:16:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:16:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:16:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:16:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271876585000001, 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:16:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:16:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271831366000001, 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:16:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:16:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271792086000001, 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:16:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:16:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271761600000001, 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:16:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:16:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:16:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271732307000001, 2024-09-21T23:16:31.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:16:31.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:31.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.498 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.499 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.499 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:31.499 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271689965, 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:16:31.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:16:31.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:16:31.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:16:31.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:16:31.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:16:31.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:16:31.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:16:31.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:16:31.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:16:31.506 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.506 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.506 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.506 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.507 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:31.507 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.271306769999999, 2024-09-21T23:16:31.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:16:31.507 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:31.507 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:31.507 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:31.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:31.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:31.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:16:31.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.508 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:31.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:31.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:31.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:31.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:31.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:31.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:31.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:16:31.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:16:31.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:31.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:16:31.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:16:31.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:31.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:16:31.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:16:31.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:16:31.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:16:31.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:31.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:16:31.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:16:31.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:31.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:16:31.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:16:31.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:31.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:31.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:31.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:16:31.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:16:31.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:31.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:31.517 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:31.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:31.520 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:31.516+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:31.620 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:31.616+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:31.653 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:31.654 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:31.654 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:31.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:31.720 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:31.716+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:31.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:31.781 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:31.781 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:31.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:31.821 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:31.816+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.746535144999999, 2024-09-21T23:16:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:16:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:16:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.681280387000001, 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:16:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:16:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 20.283938254999999, 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 20.280570566000002, 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:16:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 20.276460986, 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:16:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:16:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 20.272668065000001, 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.212509258000001, 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208752187000002, 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:16:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208725386000001, 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:16:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:16:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208699221, 2024-09-21T23:16:31.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:16:31.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208670771000001, 2024-09-21T23:16:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:16:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.957 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:16:31.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:16:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:16:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208580062999999, 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:16:31.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208548484000001, 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:16:31.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:16:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208510793999999, 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:16:31.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:16:31.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:31.978 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208480403999999, 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:16:31.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:16:31.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208436986999999, 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:16:31.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:16:31.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.208402362000001, 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:31.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:31.991 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.205148862000001, 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:31.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:16:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:31.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.20511428, 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.996 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:16:31.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:31.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.989369881, 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:32.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:32.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:16:32.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:16:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.006 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:32.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:16:32.008 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 0.125068 (2004) 2024-09-21T23:16:32.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:32.009 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:32.008+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:32.109 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:32.104+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:32.180 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:32.180 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:32.180 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:32.180 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:32.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:32.239 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:32.240+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:32.305 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:32.305 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:32.310 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:32.308+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:32.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:32.411 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:32.408+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:32.489 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726706582, 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726661325, 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726611703, 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726582007999999, 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:16:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.72654908, 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:16:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726514784000001, 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:16:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726490278, 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:32.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726462288, 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:32.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726439259999999, 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:16:32.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.72641775, 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:32.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:16:32.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:32.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726382209000001, 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:32.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:16:32.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726354572, 2024-09-21T23:16:32.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726319408, 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:32.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:32.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726284377000001, 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:32.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.562 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726248992, 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.567 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726208809999999, 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.568 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:32.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726171682, 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:32.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:32.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726131606999999, 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.578 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:32.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726107063000001, 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:32.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.587 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.726069278000001, 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:32.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:32.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:16:32.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:32.592 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:32.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:32.594 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:32.588+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:32.695 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:32.692+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:32.726 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:32.726 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:32.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:32.796 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:32.792+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:32.866 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:32.866 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:32.866 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:32.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:32.928 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:32.928+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:32.997 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:32.992+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:33.004 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.094148032, 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.093127185, 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.093067511999998, 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092977535999999, 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092927625999998, 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092871486, 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092817703000001, 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092785427000003, 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092755146999998, 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:33.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:33.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:33.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092725491000003, 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:33.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092695524, 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:33.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092617695000001, 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:33.078 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:33.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:33.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.083 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092569988000001, 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:33.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092539791999997, 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:33.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:33.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092509874999998, 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:33.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:33.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092480074999997, 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:33.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.101 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:33.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.105 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092450141999997, 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.106 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:33.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092385178999997, 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:33.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:33.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092354702999998, 2024-09-21T23:16:33.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:33.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:33.121 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.092325539000001, 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:33.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:33.126 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:33.127 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:33.129 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:33.129 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:33.124+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:33.229 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:33.228+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:33.258 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:33.259 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:33.272 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:33.327 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:33.327 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:33.329 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:33.328+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:33.343 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:33.343 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:33.360 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:33.361 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:33.379 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:33.379 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:33.396 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:33.396 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:33.399 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:33.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:33.419 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:33.419 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:33.458 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:33.456+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:33.532 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:33.528+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:33.533 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:33.533 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:33.547 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:33.638 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:33.636+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:33.720 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:33.720 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:33.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:33.735 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:33.728+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:33.834 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:33.832+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:33.903 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:33.903 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:33.918 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:33.966 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:33.964+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:34.036 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:34.036+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:34.123 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:34.123 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:34.138 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:34.136+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:34.139 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:34.238 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:34.236+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:34.263 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone18 from 0xe971200 (overlap 0x1670600) to 0xb38200 2024-09-21T23:16:34.271 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:34.271 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:34.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:34.339 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:34.336+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:34.439 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:34.436+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:34.462 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:34.462 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:34.477 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:34.570 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:34.568+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:34.641 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:34.636+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:34.645 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:34.645 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:34.662 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:34.742 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:34.740+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:34.809 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:34.809 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:34.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:34.843 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:34.840+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:34.944 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:34.940+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:34.999 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:35.048 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:35.048+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:35.127 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:35.127 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:35.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:16:35.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:35.146 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:35.144+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:35.248 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:35.244+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:35.310 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:35.310 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:35.324 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:35.349 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:35.344+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:35.439 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:35.445 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:35.445 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:35.449 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:35.444+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:35.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:35.550 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:35.548+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:35.582 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:35.599 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:35.599 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:35.617 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.87161553, 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:16:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871570273, 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:16:35.637 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:35.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871520651000001, 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:35.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871490955999999, 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871458027999999, 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.652 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:35.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:16:35.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871423732, 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:35.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:16:35.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871399226000001, 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:35.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:16:35.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871371236000002, 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871348208000001, 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:35.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:35.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871326698000001, 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:35.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:35.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871291157000002, 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:35.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871263519999999, 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:35.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:35.691 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871228356, 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:35.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:35.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:16:35.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:35.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871193325, 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:35.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:16:35.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.87115794, 2024-09-21T23:16:35.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:35.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871117758, 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871080630000002, 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:16:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871040555, 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:16:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.871016011000002, 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.870978225999998, 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:35.731 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:35.731 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:35.732 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:35.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:35.733 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:35.733 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:35.734 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:35.734 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:35.735 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:35.728+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:35.833 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:35.832+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:35.902 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:35.903 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:35.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:35.960 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:35.960+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:36.035 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:16:36.035 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:36.035+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:36.095 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:36.095 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:36.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:36.136 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:36.132+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:36.237 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:36.235+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:36.285 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.358660397000001, 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357639550000002, 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357579876999999, 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357489901000001, 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357439991, 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357383851000002, 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357330068000003, 2024-09-21T23:16:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357297791999997, 2024-09-21T23:16:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:36.326 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357267512, 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:36.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357237855999998, 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:36.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:36.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357207889000001, 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:36.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:36.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357130060000003, 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:36.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357082353000003, 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.347 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.351 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357052156999998, 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:36.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.357022239999999, 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:36.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:36.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.356992439999999, 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:36.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.356962506999999, 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:36.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:36.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:36.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.356897543999999, 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:36.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.356867068, 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:36.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.356837904000002, 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:36.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.388 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:36.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:36.389 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:36.390 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:36.391 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:36.388+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:36.463 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-21T23:16:36.491 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:36.491+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:36.522 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:36.522 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:36.522 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:36.522 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:36.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:36.579 INFO:teuthology.orchestra.run.smithi170.stdout:3075 punch from 0x6c11000 to 0x6c14a00, (0x3a00 bytes) 2024-09-21T23:16:36.592 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:36.591+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:36.597 INFO:teuthology.orchestra.run.smithi170.stdout:3076 clone 20 order 20 su 524288 sc 12 2024-09-21T23:16:36.664 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:36.664 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:36.664 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:36.664 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:36.664 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:36.678 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:36.693 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:36.691+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:36.794 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:36.792+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:36.803 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.579911685, 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:36.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:16:36.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:16:36.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.06750640200001, 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:16:36.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:16:36.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:16:36.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.067441499, 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.816 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:16:36.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.821 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.06729506000001, 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:16:36.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:16:36.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:16:36.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.067225945, 2024-09-21T23:16:36.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:36.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:16:36.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.06661362, 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:16:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:16:36.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:16:36.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.06640921499999, 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:36.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:16:36.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 110.066366429, 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:16:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:16:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.67540814100001, 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.851 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:16:36.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:16:36.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:16:36.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:36.856 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.675321631, 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:36.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:16:36.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:36.862 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.675244742, 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:16:36.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:16:36.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.67503990199999, 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.868 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:16:36.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:16:36.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.67495162, 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:16:36.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.879 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.67482079200001, 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:16:36.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:16:36.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:16:36.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.674775139, 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.886 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:16:36.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:16:36.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:36.891 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.674736712, 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.67460724999999, 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.897 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.902 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.674585608, 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.903 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.908 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.67456556400001, 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:16:36.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:16:36.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.674481572, 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:36.914 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:36.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:36.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:36.920 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:36.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:36.922 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:36.919+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:37.018 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 1] out_osds: [3] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:16:37.018 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:16:37.018 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T23:16:37.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:16:37.022 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:37.016+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:37.103 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:37.103 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:37.103 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:37.103 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:37.118 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:37.166 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:37.164+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:37.223 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:37.219+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:37.295 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:37.296 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:37.296 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:37.296 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:37.296 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:37.306 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone19 from 0xe971200 (overlap 0xe971200) to 0x4608400 2024-09-21T23:16:37.310 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:37.334 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:37.323+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:37.359 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 89, 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "309803bf-3365-47ac-8bf6-365125beceb1", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-21T23:14:32.028556+0000", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-21T23:16:37.038162+0000", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-21T23:16:02.149456+0000", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-21T23:16:14.234875+0000", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5801984, 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "noscrub", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-21T23:16:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 6, 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 6, 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "jewel", 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "squid", 2024-09-21T23:16:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:44.926374+0000", 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:16:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:16:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "15", 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:16:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:16:37.365 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:16:37.366 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 6.059999942779541, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 6.059999942779541, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T23:16:37.367 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:49.598603+0000", 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:16:37.368 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-21T23:16:37.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "19", 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:16:37.370 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 2, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 19, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:16:37.371 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:16:37.372 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2.9900000095367432, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2.9900000095367432, 2024-09-21T23:16:37.373 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.67000001668930054, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:59.367341+0000", 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:16:37.374 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-21T23:16:37.375 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "89", 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:16:37.376 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 22, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 89, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:16:37.377 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:16:37.378 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.7000000476837158, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.7000000476837158, 2024-09-21T23:16:37.379 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.82999998331069946, 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "a24df615-4443-4f27-888f-d1e4e7affbb3", 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:16:37.380 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 12, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 29, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 61, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 31, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-21T23:16:37.381 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1028371170", 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1028371170", 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1028371170", 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1028371170", 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:16:37.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d6765c2a-98b2-4a2b-8d8c-f3963575eb64", 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:16:37.386 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.387 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:16:37.388 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/2712042978", 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/2712042978", 2024-09-21T23:16:37.390 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2712042978", 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2712042978", 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "9753272a-58a9-4366-87d7-550c49134023", 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0.12506103515625, 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:16:37.391 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:16:37.392 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:16:37.392 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:16:37.392 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 74, 2024-09-21T23:16:37.392 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:16:37.392 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:16:37.392 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:16:37.393 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.393 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-21T23:16:37.393 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-21T23:16:37.394 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/1166517858", 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/1166517858", 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/1166517858", 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/1166517858", 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:16:37.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "24d987f9-f2cd-4ccd-9dc8-fea64467d92d", 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 21, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:16:37.398 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6809", 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6811", 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6814", 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6816", 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:16:37.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6822", 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6823", 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6818", 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6821", 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6811/1039392120", 2024-09-21T23:16:37.402 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6816/1039392120", 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6823/1039392120", 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6821/1039392120", 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "c1c30a49-7f26-4264-a1a9-160dc552953d", 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:16:37.403 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6808", 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6810", 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6812", 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6813", 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6819", 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6820", 2024-09-21T23:16:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6815", 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6817", 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:16:37.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6810/2764095640", 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6813/2764095640", 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6820/2764095640", 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6817/2764095640", 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f24e9341-be4e-4be2-b180-7ebd3f705e9b", 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6800", 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6801", 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6802", 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6803", 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6806", 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6807", 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:16:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6804", 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6805", 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6801/1732973431", 2024-09-21T23:16:37.413 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6803/1732973431", 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6807/1732973431", 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6805/1732973431", 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:16:37.414 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.468749+0000", 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 31 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.044837+0000", 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:16:37.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.310970+0000", 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:16:37.416 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.241402+0000", 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.877818+0000", 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:16:37.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.434052+0000", 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-21T23:16:37.418 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-09-21T23:16:37.419 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:37.420 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:37.422 INFO:tasks.thrashosds.thrasher:j is {'epoch': 89, 'fsid': '309803bf-3365-47ac-8bf6-365125beceb1', 'created': '2024-09-21T23:14:32.028556+0000', 'modified': '2024-09-21T23:16:37.038162+0000', 'last_up_change': '2024-09-21T23:16:02.149456+0000', 'last_in_change': '2024-09-21T23:16:14.234875+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:14:44.926374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:14:49.598603+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T23:14:59.367341+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 22, 'snap_epoch': 89, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a24df615-4443-4f27-888f-d1e4e7affbb3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 12, 'last_clean_end': 29, 'up_from': 61, 'up_thru': 87, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1028371170}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1028371170}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1028371170}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1028371170}]}, 'public_addr': '172.21.15.53:6817/1028371170', 'cluster_addr': '172.21.15.53:6819/1028371170', 'heartbeat_back_addr': '172.21.15.53:6823/1028371170', 'heartbeat_front_addr': '172.21.15.53:6821/1028371170', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd6765c2a-98b2-4a2b-8d8c-f3963575eb64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2712042978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2712042978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2712042978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2712042978}]}, 'public_addr': '172.21.15.53:6809/2712042978', 'cluster_addr': '172.21.15.53:6811/2712042978', 'heartbeat_back_addr': '172.21.15.53:6815/2712042978', 'heartbeat_front_addr': '172.21.15.53:6813/2712042978', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9753272a-58a9-4366-87d7-550c49134023', 'up': 1, 'in': 1, 'weight': 0.12506103515625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 1166517858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 1166517858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 1166517858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 1166517858}]}, 'public_addr': '172.21.15.53:6801/1166517858', 'cluster_addr': '172.21.15.53:6803/1166517858', 'heartbeat_back_addr': '172.21.15.53:6807/1166517858', 'heartbeat_front_addr': '172.21.15.53:6805/1166517858', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '24d987f9-f2cd-4ccd-9dc8-fea64467d92d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6809', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 1039392120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6816', 'nonce': 1039392120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6823', 'nonce': 1039392120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 1039392120}]}, 'public_addr': '172.21.15.144:6811/1039392120', 'cluster_addr': '172.21.15.144:6816/1039392120', 'heartbeat_back_addr': '172.21.15.144:6823/1039392120', 'heartbeat_front_addr': '172.21.15.144:6821/1039392120', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'c1c30a49-7f26-4264-a1a9-160dc552953d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6810', 'nonce': 2764095640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 2764095640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2764095640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6815', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2764095640}]}, 'public_addr': '172.21.15.144:6810/2764095640', 'cluster_addr': '172.21.15.144:6813/2764095640', 'heartbeat_back_addr': '172.21.15.144:6820/2764095640', 'heartbeat_front_addr': '172.21.15.144:6817/2764095640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f24e9341-be4e-4be2-b180-7ebd3f705e9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 1732973431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 1732973431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 1732973431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 1732973431}]}, 'public_addr': '172.21.15.144:6801/1732973431', 'cluster_addr': '172.21.15.144:6803/1732973431', 'heartbeat_back_addr': '172.21.15.144:6807/1732973431', 'heartbeat_front_addr': '172.21.15.144:6805/1732973431', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.468749+0000', 'dead_epoch': 31}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.044837+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.310970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.241402+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.877818+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.434052+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:16:37.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:16:37.463 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:37.463+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:37.492 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.536454548999998, 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:37.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:16:37.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:16:37.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 62.536187564999999, 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:37.498 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:16:37.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.503 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923859019999998, 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:37.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:16:37.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:16:37.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923816321, 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:16:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923777700999999, 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:16:37.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:16:37.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923745829000001, 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.516 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:16:37.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.520 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923712604999999, 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.524 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923677529999999, 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:16:37.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923632571999999, 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:16:37.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:16:37.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.923180505000001, 2024-09-21T23:16:37.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:37.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:16:37.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.922466025999999, 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.538 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:16:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.922424956, 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.921967838, 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:16:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.921731159, 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:16:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:16:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.921315578000002, 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:16:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.920637918000001, 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:16:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.920146629000001, 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:16:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.919889731000001, 2024-09-21T23:16:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:16:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:16:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26034 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7325696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.919426649999998, 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086107304999999995, 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26034 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571018+0000", 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018324500000000001 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571023+0000", 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9009999999999999e-06 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571048+0000", 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4454e-05 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571058+0000", 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0145e-05 2024-09-21T23:16:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656654+0000", 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085595579000000005 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656758+0000", 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010446999999999999 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656942+0000", 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183511 2024-09-21T23:16:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26035 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6836224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.919186265, 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086157396999999997, 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26035 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571805+0000", 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000730326 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571812+0000", 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571833+0000", 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1506999999999999e-05 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571846+0000", 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2330999999999999e-05 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657009+0000", 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085162977000000001 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657091+0000", 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2514000000000003e-05 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657232+0000", 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140858 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:37.581 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:37.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:37.583 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:37.579+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:37.683 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:37.683+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:37.730 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:37.730 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:37.730 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:37.730 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:37.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:37.753 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:37.798 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:37.796+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:37.823 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:16:37.823 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:16:37.844 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-21T23:16:37.040131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376641851,"num_objects":1020,"num_object_clones":292,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":466,"num_read":177059,"num_read_kb":459203,"num_write":12318,"num_write_kb":71306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1971,"num_bytes_recovered":23485760,"num_keys_recovered":270,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10879,"ondisk_log_size":10879,"up":82,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":279880,"kb_used_data":57552,"kb_used_omap":11,"kb_used_meta":222196,"kb_avail":471579320,"statfs":{"total":483183820800,"available":482897223680,"internally_reserved":0,"allocated":58933248,"data_stored":66970647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12231,"internal_metadata":227528761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30195833,"num_objects":226,"num_object_clones":115,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":115,"num_read":9629,"num_read_kb":24672,"num_write":603,"num_write_kb":2126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":6404166,"num_keys_recovered":71,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.043955"},"pg_stats":[{"pgid":"3.1f","version":"86'319","reported_seq":10820,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.051323+0000","last_change":"2024-09-21T23:16:36.051323+0000","last_active":"2024-09-21T23:16:36.051323+0000","last_peered":"2024-09-21T23:16:36.051323+0000","last_clean":"2024-09-21T23:16:36.051323+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:16:36.051323+0000","last_undegraded":"2024-09-21T23:16:36.051323+0000","last_fullsized":"2024-09-21T23:16:36.051323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:29.527116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":17920601,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":19,"num_read":10625,"num_read_kb":14274,"num_write":403,"num_write_kb":1210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"86'221","reported_seq":1376,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.505881+0000","last_change":"2024-09-21T23:16:34.444511+0000","last_active":"2024-09-21T23:16:34.505881+0000","last_peered":"2024-09-21T23:16:34.505881+0000","last_clean":"2024-09-21T23:16:31.119621+0000","last_became_active":"2024-09-21T23:16:34.028755+0000","last_became_peered":"2024-09-21T23:16:34.028755+0000","last_unstale":"2024-09-21T23:16:34.505881+0000","last_undegraded":"2024-09-21T23:16:34.505881+0000","last_fullsized":"2024-09-21T23:16:34.008730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":9241652,"num_objects":30,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":746,"num_read_kb":9357,"num_write":206,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":["1","2","5"],"object_location_counts":[{"shards":"1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"86'145","reported_seq":1436,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.298995+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:16:34.298995+0000","last_peered":"2024-09-21T23:16:34.298995+0000","last_clean":"2024-09-21T23:16:34.298995+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:16:34.298995+0000","last_undegraded":"2024-09-21T23:16:34.298995+0000","last_fullsized":"2024-09-21T23:16:34.298995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:04:21.741029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":6862848,"num_objects":30,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":15,"num_read":1175,"num_read_kb":6942,"num_write":133,"num_write_kb":2087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"86'283","reported_seq":4588,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.861452+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:16:34.861452+0000","last_peered":"2024-09-21T23:16:34.861452+0000","last_clean":"2024-09-21T23:16:34.861452+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:16:34.861452+0000","last_undegraded":"2024-09-21T23:16:34.861452+0000","last_fullsized":"2024-09-21T23:16:34.861452+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:15.111467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":17249872,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":5038,"num_read_kb":20409,"num_write":254,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'270","reported_seq":2136,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.043666+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:16:36.043666+0000","last_peered":"2024-09-21T23:16:36.043666+0000","last_clean":"2024-09-21T23:16:36.043666+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:16:36.043666+0000","last_undegraded":"2024-09-21T23:16:36.043666+0000","last_fullsized":"2024-09-21T23:16:36.043666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:11:13.108533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":7343683,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":2725,"num_read_kb":11628,"num_write":281,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"86'343","reported_seq":5848,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.869830+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:16:34.869830+0000","last_peered":"2024-09-21T23:16:34.869830+0000","last_clean":"2024-09-21T23:16:34.869830+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:16:34.869830+0000","last_undegraded":"2024-09-21T23:16:34.869830+0000","last_fullsized":"2024-09-21T23:16:34.869830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:31.439108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":8448547,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":7062,"num_read_kb":17071,"num_write":255,"num_write_kb":856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"86'185","reported_seq":1358,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.849157+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:16:34.849157+0000","last_peered":"2024-09-21T23:16:34.849157+0000","last_clean":"2024-09-21T23:16:34.849157+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:16:34.849157+0000","last_undegraded":"2024-09-21T23:16:34.849157+0000","last_fullsized":"2024-09-21T23:16:34.849157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:09.503697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":10276384,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":1011,"num_read_kb":6686,"num_write":172,"num_write_kb":1230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"86'393","reported_seq":11333,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.048063+0000","last_change":"2024-09-21T23:16:36.046053+0000","last_active":"2024-09-21T23:16:36.048063+0000","last_peered":"2024-09-21T23:16:36.048063+0000","last_clean":"2024-09-21T23:16:36.048063+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:16:36.048063+0000","last_undegraded":"2024-09-21T23:16:36.048063+0000","last_fullsized":"2024-09-21T23:16:36.048063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:25.671262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":24997436,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":11735,"num_read_kb":25412,"num_write":538,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"86'210","reported_seq":9028,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.152143+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:16:36.152143+0000","last_peered":"2024-09-21T23:16:36.152143+0000","last_clean":"2024-09-21T23:16:36.152143+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:16:36.152143+0000","last_undegraded":"2024-09-21T23:16:36.152143+0000","last_fullsized":"2024-09-21T23:16:36.152143+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:29:25.593974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":13400128,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":8671,"num_read_kb":16110,"num_write":191,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"86'384","reported_seq":2827,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.399941+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:16:34.399941+0000","last_peered":"2024-09-21T23:16:34.399941+0000","last_clean":"2024-09-21T23:16:34.399941+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:16:34.399941+0000","last_undegraded":"2024-09-21T23:16:34.399941+0000","last_fullsized":"2024-09-21T23:16:34.399941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:13.300296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":33834658,"num_objects":66,"num_object_clones":24,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":36,"num_read":2890,"num_read_kb":8164,"num_write":465,"num_write_kb":2682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'140","reported_seq":3009,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.042807+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:16:36.042807+0000","last_peered":"2024-09-21T23:16:36.042807+0000","last_clean":"2024-09-21T23:16:36.042807+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:16:36.042807+0000","last_undegraded":"2024-09-21T23:16:36.042807+0000","last_fullsized":"2024-09-21T23:16:36.042807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:46:50.763981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623296,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":2737,"num_read_kb":24503,"num_write":137,"num_write_kb":2876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"86'524","reported_seq":1953,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.518923+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:16:34.518923+0000","last_peered":"2024-09-21T23:16:34.518923+0000","last_clean":"2024-09-21T23:16:34.518923+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:16:34.518923+0000","last_undegraded":"2024-09-21T23:16:34.518923+0000","last_fullsized":"2024-09-21T23:16:34.518923+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:37:16.998356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9595551,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":3091,"num_read_kb":12089,"num_write":757,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:33.999989+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:16:33.999989+0000","last_peered":"2024-09-21T23:16:33.999989+0000","last_clean":"2024-09-21T23:16:33.999989+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:16:33.999989+0000","last_undegraded":"2024-09-21T23:16:33.999989+0000","last_fullsized":"2024-09-21T23:16:33.999989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'574","reported_seq":17716,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.152367+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:16:36.152367+0000","last_peered":"2024-09-21T23:16:36.152367+0000","last_clean":"2024-09-21T23:16:36.152367+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:16:36.152367+0000","last_undegraded":"2024-09-21T23:16:36.152367+0000","last_fullsized":"2024-09-21T23:16:36.152367+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:39.890689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":27026515,"num_objects":34,"num_object_clones":11,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":18849,"num_read_kb":26337,"num_write":910,"num_write_kb":3109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.011855+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:16:34.011855+0000","last_peered":"2024-09-21T23:16:34.011855+0000","last_clean":"2024-09-21T23:16:34.011855+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:16:34.011855+0000","last_undegraded":"2024-09-21T23:16:34.011855+0000","last_fullsized":"2024-09-21T23:16:34.011855+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"86'1506","reported_seq":4563,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.791015+0000","last_change":"2024-09-21T23:16:34.791015+0000","last_active":"2024-09-21T23:16:34.791015+0000","last_peered":"2024-09-21T23:16:34.791015+0000","last_clean":"2024-09-21T23:16:31.163798+0000","last_became_active":"2024-09-21T23:16:34.057114+0000","last_became_peered":"2024-09-21T23:16:34.057114+0000","last_unstale":"2024-09-21T23:16:34.791015+0000","last_undegraded":"2024-09-21T23:16:34.791015+0000","last_fullsized":"2024-09-21T23:16:34.009773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1506,"log_dups_size":0,"ondisk_log_size":1506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":9421738,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":10,"num_read":9852,"num_read_kb":29377,"num_write":2680,"num_write_kb":5451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1],"avail_no_missing":["1","2","4"],"object_location_counts":[{"shards":"1","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.042729+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:16:36.042729+0000","last_peered":"2024-09-21T23:16:36.042729+0000","last_clean":"2024-09-21T23:16:36.042729+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:16:36.042729+0000","last_undegraded":"2024-09-21T23:16:36.042729+0000","last_fullsized":"2024-09-21T23:16:36.042729+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"86'267","reported_seq":1107,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.515573+0000","last_change":"2024-09-21T23:16:34.515855+0000","last_active":"2024-09-21T23:16:34.515573+0000","last_peered":"2024-09-21T23:16:34.515573+0000","last_clean":"2024-09-21T23:16:31.074953+0000","last_became_active":"2024-09-21T23:16:34.048836+0000","last_became_peered":"2024-09-21T23:16:34.048836+0000","last_unstale":"2024-09-21T23:16:34.515573+0000","last_undegraded":"2024-09-21T23:16:34.011911+0000","last_fullsized":"2024-09-21T23:16:34.009840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":5021219,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":1612,"num_read_kb":2631,"num_write":314,"num_write_kb":780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"1","objects":28}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.012566+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:16:34.012566+0000","last_peered":"2024-09-21T23:16:34.012566+0000","last_clean":"2024-09-21T23:16:34.012566+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:16:34.012566+0000","last_undegraded":"2024-09-21T23:16:34.012566+0000","last_fullsized":"2024-09-21T23:16:34.012566+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"86'318","reported_seq":2622,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.520260+0000","last_change":"2024-09-21T23:15:03.321597+0000","last_active":"2024-09-21T23:16:34.520260+0000","last_peered":"2024-09-21T23:16:34.520260+0000","last_clean":"2024-09-21T23:16:34.520260+0000","last_became_active":"2024-09-21T23:15:01.329113+0000","last_became_peered":"2024-09-21T23:15:01.329113+0000","last_unstale":"2024-09-21T23:16:34.520260+0000","last_undegraded":"2024-09-21T23:16:34.520260+0000","last_fullsized":"2024-09-21T23:16:34.520260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:18:08.955476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":12211527,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":3536,"num_read_kb":15396,"num_write":358,"num_write_kb":1568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"86'219","reported_seq":2906,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.518840+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:16:34.518840+0000","last_peered":"2024-09-21T23:16:34.518840+0000","last_clean":"2024-09-21T23:16:34.518840+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:16:34.518840+0000","last_undegraded":"2024-09-21T23:16:34.518840+0000","last_fullsized":"2024-09-21T23:16:34.518840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:51.970111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":11943454,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":2591,"num_read_kb":14453,"num_write":219,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":175,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.003582+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:16:34.003582+0000","last_peered":"2024-09-21T23:16:34.003582+0000","last_clean":"2024-09-21T23:16:34.003582+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:16:34.003582+0000","last_undegraded":"2024-09-21T23:16:34.003582+0000","last_fullsized":"2024-09-21T23:16:34.003582+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"86'228","reported_seq":2959,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.688962+0000","last_change":"2024-09-21T23:16:34.263927+0000","last_active":"2024-09-21T23:16:34.688962+0000","last_peered":"2024-09-21T23:16:34.688962+0000","last_clean":"2024-09-21T23:16:31.121418+0000","last_became_active":"2024-09-21T23:16:34.027957+0000","last_became_peered":"2024-09-21T23:16:34.027957+0000","last_unstale":"2024-09-21T23:16:34.688962+0000","last_undegraded":"2024-09-21T23:16:34.688962+0000","last_fullsized":"2024-09-21T23:16:34.008624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":4406900,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":2521,"num_read_kb":8021,"num_write":192,"num_write_kb":731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1],"avail_no_missing":["1","2","4"],"object_location_counts":[{"shards":"1","objects":28}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.008889+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:16:34.008889+0000","last_peered":"2024-09-21T23:16:34.008889+0000","last_clean":"2024-09-21T23:16:34.008889+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:16:34.008889+0000","last_undegraded":"2024-09-21T23:16:34.008889+0000","last_fullsized":"2024-09-21T23:16:34.008889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"86'258","reported_seq":2691,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.042289+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:16:36.042289+0000","last_peered":"2024-09-21T23:16:36.042289+0000","last_clean":"2024-09-21T23:16:36.042289+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:16:36.042289+0000","last_undegraded":"2024-09-21T23:16:36.042289+0000","last_fullsized":"2024-09-21T23:16:36.042289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:47:07.882033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":8306724,"num_objects":37,"num_object_clones":13,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":2357,"num_read_kb":10187,"num_write":247,"num_write_kb":4079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.026001+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:16:34.026001+0000","last_peered":"2024-09-21T23:16:34.026001+0000","last_clean":"2024-09-21T23:16:34.026001+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:16:34.026001+0000","last_undegraded":"2024-09-21T23:16:34.026001+0000","last_fullsized":"2024-09-21T23:16:34.026001+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"86'249","reported_seq":10662,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.426684+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:16:34.426684+0000","last_peered":"2024-09-21T23:16:34.426684+0000","last_clean":"2024-09-21T23:16:34.426684+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:16:34.426684+0000","last_undegraded":"2024-09-21T23:16:34.426684+0000","last_fullsized":"2024-09-21T23:16:34.426684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:21.696611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":16807536,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":11167,"num_read_kb":28111,"num_write":243,"num_write_kb":2118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":162,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.005723+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:16:34.005723+0000","last_peered":"2024-09-21T23:16:34.005723+0000","last_clean":"2024-09-21T23:16:34.005723+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:16:34.005723+0000","last_undegraded":"2024-09-21T23:16:34.005723+0000","last_fullsized":"2024-09-21T23:16:34.005723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":256,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.042497+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:16:36.042497+0000","last_peered":"2024-09-21T23:16:36.042497+0000","last_clean":"2024-09-21T23:16:36.042497+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:16:36.042497+0000","last_undegraded":"2024-09-21T23:16:36.042497+0000","last_fullsized":"2024-09-21T23:16:36.042497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"86'546","reported_seq":10173,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.632021+0000","last_change":"2024-09-21T23:16:34.632021+0000","last_active":"2024-09-21T23:16:34.632021+0000","last_peered":"2024-09-21T23:16:34.632021+0000","last_clean":"2024-09-21T23:16:31.195801+0000","last_became_active":"2024-09-21T23:16:34.047671+0000","last_became_peered":"2024-09-21T23:16:34.047671+0000","last_unstale":"2024-09-21T23:16:34.632021+0000","last_undegraded":"2024-09-21T23:16:34.632021+0000","last_fullsized":"2024-09-21T23:16:34.009175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":5524000,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":10603,"num_read_kb":14915,"num_write":329,"num_write_kb":1497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":["1","2","5"],"object_location_counts":[{"shards":"1","objects":32}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'347","reported_seq":3105,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.448989+0000","last_change":"2024-09-21T23:16:34.108629+0000","last_active":"2024-09-21T23:16:34.448989+0000","last_peered":"2024-09-21T23:16:34.448989+0000","last_clean":"2024-09-21T23:16:31.124357+0000","last_became_active":"2024-09-21T23:16:34.012198+0000","last_became_peered":"2024-09-21T23:16:34.012198+0000","last_unstale":"2024-09-21T23:16:34.448989+0000","last_undegraded":"2024-09-21T23:16:34.448989+0000","last_fullsized":"2024-09-21T23:16:33.999756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":6558425,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":4358,"num_read_kb":8072,"num_write":256,"num_write_kb":926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4],"avail_no_missing":["4","2","5"],"object_location_counts":[{"shards":"4","objects":35}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'185","reported_seq":724,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.518634+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:16:34.518634+0000","last_peered":"2024-09-21T23:16:34.518634+0000","last_clean":"2024-09-21T23:16:34.518634+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:16:34.518634+0000","last_undegraded":"2024-09-21T23:16:34.518634+0000","last_fullsized":"2024-09-21T23:16:34.518634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:58.270682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4407312,"num_objects":30,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":15,"num_read":301,"num_read_kb":1498,"num_write":171,"num_write_kb":723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"86'425","reported_seq":6091,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.874617+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:16:34.874617+0000","last_peered":"2024-09-21T23:16:34.874617+0000","last_clean":"2024-09-21T23:16:34.874617+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:16:34.874617+0000","last_undegraded":"2024-09-21T23:16:34.874617+0000","last_fullsized":"2024-09-21T23:16:34.874617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:01:25.337956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":8956447,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":7574,"num_read_kb":16196,"num_write":287,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"86'312","reported_seq":1542,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.856019+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:16:34.856019+0000","last_peered":"2024-09-21T23:16:34.856019+0000","last_clean":"2024-09-21T23:16:34.856019+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:16:34.856019+0000","last_undegraded":"2024-09-21T23:16:34.856019+0000","last_fullsized":"2024-09-21T23:16:34.856019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:21:47.252577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":14159405,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":1827,"num_read_kb":3243,"num_write":281,"num_write_kb":1314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"86'208","reported_seq":3450,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.738897+0000","last_change":"2024-09-21T23:16:34.221492+0000","last_active":"2024-09-21T23:16:34.738897+0000","last_peered":"2024-09-21T23:16:34.738897+0000","last_clean":"2024-09-21T23:16:31.128298+0000","last_became_active":"2024-09-21T23:16:34.036249+0000","last_became_peered":"2024-09-21T23:16:34.036249+0000","last_unstale":"2024-09-21T23:16:34.738897+0000","last_undegraded":"2024-09-21T23:16:34.738897+0000","last_fullsized":"2024-09-21T23:16:33.999867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6431500,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":3325,"num_read_kb":23578,"num_write":237,"num_write_kb":1185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":["4","0","2"],"object_location_counts":[{"shards":"4","objects":25}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"86'258","reported_seq":2188,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.333349+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:16:34.333349+0000","last_peered":"2024-09-21T23:16:34.333349+0000","last_clean":"2024-09-21T23:16:34.333349+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:16:34.333349+0000","last_undegraded":"2024-09-21T23:16:34.333349+0000","last_fullsized":"2024-09-21T23:16:34.333349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:24:10.188979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":7965262,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":2957,"num_read_kb":8848,"num_write":259,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"83'250","reported_seq":3214,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.881187+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:16:34.881187+0000","last_peered":"2024-09-21T23:16:34.881187+0000","last_clean":"2024-09-21T23:16:34.881187+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:16:34.881187+0000","last_undegraded":"2024-09-21T23:16:34.881187+0000","last_fullsized":"2024-09-21T23:16:34.881187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:48.921210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":5008400,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":4263,"num_read_kb":8463,"num_write":183,"num_write_kb":685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"86'228","reported_seq":2466,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.508841+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:16:34.508841+0000","last_peered":"2024-09-21T23:16:34.508841+0000","last_clean":"2024-09-21T23:16:34.508841+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:16:34.508841+0000","last_undegraded":"2024-09-21T23:16:34.508841+0000","last_fullsized":"2024-09-21T23:16:34.508841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:45:51.952500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":12374059,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2024,"num_read_kb":11464,"num_write":211,"num_write_kb":1581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"86'373","reported_seq":1988,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T23:16:34.325550+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:16:34.325550+0000","last_peered":"2024-09-21T23:16:34.325550+0000","last_clean":"2024-09-21T23:16:34.325550+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:16:34.325550+0000","last_undegraded":"2024-09-21T23:16:34.325550+0000","last_fullsized":"2024-09-21T23:16:34.325550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:53:16.292534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5056037,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":2565,"num_read_kb":6062,"num_write":527,"num_write_kb":1294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"86'392","reported_seq":17826,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T23:16:36.188690+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:16:36.188690+0000","last_peered":"2024-09-21T23:16:36.188690+0000","last_clean":"2024-09-21T23:16:36.188690+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:16:36.188690+0000","last_undegraded":"2024-09-21T23:16:36.188690+0000","last_fullsized":"2024-09-21T23:16:36.188690+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:10.230644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":22563388,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":19667,"num_read_kb":32915,"num_write":280,"num_write_kb":2034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"86'199","reported_seq":7936,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-21T23:16:34.518931+0000","last_change":"2024-09-21T23:16:34.115232+0000","last_active":"2024-09-21T23:16:34.518931+0000","last_peered":"2024-09-21T23:16:34.518931+0000","last_clean":"2024-09-21T23:16:31.225992+0000","last_became_active":"2024-09-21T23:16:34.009870+0000","last_became_peered":"2024-09-21T23:16:34.009870+0000","last_unstale":"2024-09-21T23:16:34.518931+0000","last_undegraded":"2024-09-21T23:16:34.518931+0000","last_fullsized":"2024-09-21T23:16:34.002728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9106960,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":7488,"num_read_kb":16691,"num_write":181,"num_write_kb":805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5],"avail_no_missing":["5","2","4"],"object_location_counts":[{"shards":"5","objects":32}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":376051464,"num_objects":1016,"num_object_clones":292,"num_object_copies":2032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1016,"num_whiteouts":466,"num_read":176943,"num_read_kb":459103,"num_write":12157,"num_write_kb":68888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1964,"num_bytes_recovered":22304986,"num_keys_recovered":270,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":56344576,"data_stored":67884646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":10759,"ondisk_log_size":10759,"up":64,"acting":56,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57936,"kb_used_data":14264,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94313904,"statfs":{"total":96636764160,"available":96577437696,"internally_reserved":0,"allocated":14606336,"data_stored":17632339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44008,"kb_used_data":12820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327832,"statfs":{"total":96636764160,"available":96591699968,"internally_reserved":0,"allocated":13127680,"data_stored":15872303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607578,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607578,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61288,"kb_used_data":7884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310552,"statfs":{"total":96636764160,"available":96574005248,"internally_reserved":0,"allocated":8073216,"data_stored":9139225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607579,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69244,"kb_used_data":11680,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94302596,"statfs":{"total":96636764160,"available":96565858304,"internally_reserved":0,"allocated":11960320,"data_stored":12377045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005064,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47404,"kb_used_data":10904,"kb_used_omap":5,"kb_used_meta":36474,"kb_avail":94324436,"statfs":{"total":96636764160,"available":96588222464,"internally_reserved":0,"allocated":11165696,"data_stored":11949735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":37349669},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9555968,"data_stored":10941892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11689984,"data_stored":13277924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7069696,"data_stored":8726925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14163968,"data_stored":17474637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13860864,"data_stored":17463252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:16:37.846 INFO:tasks.thrashosds.thrasher:Setting 3.13 to [4, 3, 1, 0] 2024-09-21T23:16:37.846 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.13', '4', '3', '1', '0'] 2024-09-21T23:16:37.847 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.13 4 3 1 0 2024-09-21T23:16:37.891 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:37.891+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:37.894 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:37.894 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:37.894 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:37.894 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:37.894 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:37.906 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:37.907 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:37.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:37.924 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:37.924 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:37.941 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:37.961 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:37.961 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:37.982 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:37.982 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:37.998 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:37.996+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:38.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:38.000 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:38.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:16:38.075 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:38.075 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:38.090 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:38.115 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:38.115+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:38.188 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:38.184+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:38.242 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:38.242 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:38.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:38.299 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:38.299+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:38.377 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:38.377 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:38.389 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:38.384+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:38.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:38.490 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:38.488+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:38.557 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:38.557 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:38.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:38.615 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:38.615+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:38.692 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:38.688+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:38.697 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:38.698 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:38.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:38.793 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:38.788+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:38.873 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-21T23:16:38.881 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:38.882 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:38.894 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:38.891+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:38.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:38.995 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:38.992+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:39.042 INFO:teuthology.orchestra.run.smithi053.stderr:set 3.13 pg_upmap_items mapping to [4->3,1->0] 2024-09-21T23:16:39.047 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:16:39.087 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:39.087 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:39.095 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:39.095+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:39.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:39.123 INFO:teuthology.orchestra.run.smithi170.stdout:3079 write 0x44b9a00 thru 0x44ba7d8 (0xdd9 bytes) 2024-09-21T23:16:39.129 INFO:teuthology.orchestra.run.smithi170.stdout:3081 write 0x32bd600 thru 0x32c21c7 (0x4bc8 bytes) 2024-09-21T23:16:39.130 INFO:teuthology.orchestra.run.smithi170.stdout:3082 write 0x1ab4a00 thru 0x1ac169d (0xcc9e bytes) 2024-09-21T23:16:39.132 INFO:teuthology.orchestra.run.smithi170.stdout:3083 read 0xc161800 thru 0xc163f09 (0x270a bytes) 2024-09-21T23:16:39.137 INFO:teuthology.orchestra.run.smithi170.stdout:3084 punch from 0xcaffc00 to 0xcb05c00, (0x6000 bytes) 2024-09-21T23:16:39.188 INFO:teuthology.orchestra.run.smithi170.stdout:3088 read 0x7784000 thru 0x7786d15 (0x2d16 bytes) 2024-09-21T23:16:39.188 INFO:teuthology.orchestra.run.smithi170.stdout:3090 read 0x425b200 thru 0x426238e (0x718f bytes) 2024-09-21T23:16:39.189 INFO:teuthology.orchestra.run.smithi170.stdout:3091 punch from 0x99d1a00 to 0x99d6600, (0x4c00 bytes) 2024-09-21T23:16:39.196 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:39.192+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:39.197 INFO:teuthology.orchestra.run.smithi170.stdout:3092 trunc from 0xe971200 to 0xe983c00 2024-09-21T23:16:39.276 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:39.276 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:39.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:39.329 INFO:teuthology.orchestra.run.smithi170.stdout:3094 trunc from 0xe983c00 to 0x1b92200 2024-09-21T23:16:39.339 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:39.339+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:39.398 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:39.395+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:39.438 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:39.438 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:39.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:39.479 INFO:teuthology.orchestra.run.smithi170.stdout:3095 trunc from 0x1b92200 to 0xedbf000 2024-09-21T23:16:39.498 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:39.496+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:39.507 INFO:teuthology.orchestra.run.smithi170.stdout:3096 read 0xa092600 thru 0xa093a07 (0x1408 bytes) 2024-09-21T23:16:39.508 INFO:teuthology.orchestra.run.smithi170.stdout:3097 write 0xdc62e00 thru 0xdc7219c (0xf39d bytes) 2024-09-21T23:16:39.509 INFO:teuthology.orchestra.run.smithi170.stdout:3098 read 0xc9bda00 thru 0xc9c49aa (0x6fab bytes) 2024-09-21T23:16:39.509 INFO:teuthology.orchestra.run.smithi170.stdout:3100 read 0xd2a3e00 thru 0xd2ab450 (0x7651 bytes) 2024-09-21T23:16:39.523 INFO:teuthology.orchestra.run.smithi170.stdout:3102 trunc from 0xedbf000 to 0x7098600 2024-09-21T23:16:39.559 INFO:teuthology.orchestra.run.smithi170.stdout:3103 read 0x44c1800 thru 0x44c93f4 (0x7bf5 bytes) 2024-09-21T23:16:39.561 INFO:teuthology.orchestra.run.smithi170.stdout:3104 write 0x2f4e000 thru 0x2f5d445 (0xf446 bytes) 2024-09-21T23:16:39.580 INFO:teuthology.orchestra.run.smithi170.stdout:3105 trunc from 0x7098600 to 0x14a4800 2024-09-21T23:16:39.599 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:39.595+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:39.655 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:39.655 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:39.677 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:39.719 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:39.719+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:39.792 INFO:teuthology.orchestra.run.smithi170.stdout:3106 write 0xed46800 thru 0xed4b2d4 (0x4ad5 bytes) 2024-09-21T23:16:39.795 INFO:teuthology.orchestra.run.smithi170.stdout:3109 read 0xed1b000 thru 0xed27fa5 (0xcfa6 bytes) 2024-09-21T23:16:39.795 INFO:teuthology.orchestra.run.smithi170.stdout:3111 read 0x3fddc00 thru 0x3fe5b06 (0x7f07 bytes) 2024-09-21T23:16:39.795 INFO:teuthology.orchestra.run.smithi170.stdout:3112 write 0x257f000 thru 0x258baaf (0xcab0 bytes) 2024-09-21T23:16:39.802 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:39.796+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:39.815 INFO:teuthology.orchestra.run.smithi170.stdout:3113 trunc from 0xed4b400 to 0x69ff000 2024-09-21T23:16:39.818 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:39.818 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:39.836 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:39.862 INFO:teuthology.orchestra.run.smithi170.stdout:3115 read 0x52e8e00 thru 0x52f1d48 (0x8f49 bytes) 2024-09-21T23:16:39.864 INFO:teuthology.orchestra.run.smithi170.stdout:3119 read 0x1c5a400 thru 0x1c5b3f5 (0xff6 bytes) 2024-09-21T23:16:39.864 INFO:teuthology.orchestra.run.smithi170.stdout:3121 read 0x345d600 thru 0x346a7ac (0xd1ad bytes) 2024-09-21T23:16:39.875 INFO:teuthology.orchestra.run.smithi170.stdout:3125 trunc from 0x69ff000 to 0x198d600 2024-09-21T23:16:39.902 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:39.899+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:39.913 INFO:teuthology.orchestra.run.smithi170.stdout:3127 trunc from 0x198d600 to 0x199ce00 2024-09-21T23:16:39.941 INFO:teuthology.orchestra.run.smithi170.stdout:3128 read 0xa03800 thru 0xa06c7c (0x347d bytes) 2024-09-21T23:16:39.943 INFO:teuthology.orchestra.run.smithi170.stdout:3130 read 0x340800 thru 0x3484e3 (0x7ce4 bytes) 2024-09-21T23:16:39.945 INFO:teuthology.orchestra.run.smithi170.stdout:3135 read 0x295600 thru 0x29e45c (0x8e5d bytes) 2024-09-21T23:16:39.947 INFO:teuthology.orchestra.run.smithi170.stdout:3140 punch from 0x6eb400 to 0x6f4800, (0x9400 bytes) 2024-09-21T23:16:39.949 INFO:teuthology.orchestra.run.smithi170.stdout:3141 read 0x130aa00 thru 0x1311d31 (0x7332 bytes) 2024-09-21T23:16:39.987 INFO:teuthology.orchestra.run.smithi170.stdout:3142 trunc from 0x199ce00 to 0xc33aa00 2024-09-21T23:16:40.002 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:39.999+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:40.011 INFO:teuthology.orchestra.run.smithi170.stdout:3143 read 0x7e81200 thru 0x7e884ce (0x72cf bytes) 2024-09-21T23:16:40.013 INFO:teuthology.orchestra.run.smithi170.stdout:3144 write 0x97cc000 thru 0x97ce5cc (0x25cd bytes) 2024-09-21T23:16:40.013 INFO:teuthology.orchestra.run.smithi170.stdout:3145 punch from 0x7baec00 to 0x7bbe200, (0xf600 bytes) 2024-09-21T23:16:40.013 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:40.013 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:40.014 INFO:teuthology.orchestra.run.smithi170.stdout:3147 read 0x75be400 thru 0x75bf3bf (0xfc0 bytes) 2024-09-21T23:16:40.014 INFO:teuthology.orchestra.run.smithi170.stdout:3150 read 0x7667e00 thru 0x766a7ec (0x29ed bytes) 2024-09-21T23:16:40.014 INFO:teuthology.orchestra.run.smithi170.stdout:3152 punch from 0x8a83a00 to 0x8a8ee00, (0xb400 bytes) 2024-09-21T23:16:40.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:40.027 INFO:teuthology.orchestra.run.smithi170.stdout:3153 trunc from 0xc33aa00 to 0x6db5600 2024-09-21T23:16:40.029 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:40.057 INFO:teuthology.orchestra.run.smithi170.stdout:3155 read 0x3b39600 thru 0x3b48bf1 (0xf5f2 bytes) 2024-09-21T23:16:40.059 INFO:teuthology.orchestra.run.smithi170.stdout:3156 read 0x5d6ae00 thru 0x5d738c3 (0x8ac4 bytes) 2024-09-21T23:16:40.059 INFO:teuthology.orchestra.run.smithi170.stdout:3157 write 0x2555800 thru 0x2556b1c (0x131d bytes) 2024-09-21T23:16:40.081 INFO:teuthology.orchestra.run.smithi170.stdout:3158 write 0x837bc00 thru 0x8381f04 (0x6305 bytes) 2024-09-21T23:16:40.086 INFO:teuthology.orchestra.run.smithi170.stdout:3162 trunc from 0x8382000 to 0x701c800 2024-09-21T23:16:40.103 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:40.099+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:40.124 INFO:teuthology.orchestra.run.smithi170.stdout:3166 trunc from 0x701c800 to 0xa2bbe00 2024-09-21T23:16:40.141 INFO:teuthology.orchestra.run.smithi170.stdout:3170 punch from 0x983b800 to 0x983c200, (0xa00 bytes) 2024-09-21T23:16:40.143 INFO:teuthology.orchestra.run.smithi170.stdout:3173 read 0x5515000 thru 0x551afa9 (0x5faa bytes) 2024-09-21T23:16:40.143 INFO:teuthology.orchestra.run.smithi170.stdout:3174 write 0x5409400 thru 0x54123b7 (0x8fb8 bytes) 2024-09-21T23:16:40.143 INFO:teuthology.orchestra.run.smithi170.stdout:3175 read 0x76a8400 thru 0x76b3b63 (0xb764 bytes) 2024-09-21T23:16:40.144 INFO:teuthology.orchestra.run.smithi170.stdout:3176 write 0x3d3cc00 thru 0x3d4a0bd (0xd4be bytes) 2024-09-21T23:16:40.144 INFO:teuthology.orchestra.run.smithi170.stdout:3177 read 0x6652c00 thru 0x665ba1d (0x8e1e bytes) 2024-09-21T23:16:40.161 INFO:teuthology.orchestra.run.smithi170.stdout:3179 trunc from 0xa2bbe00 to 0x3eb3400 2024-09-21T23:16:40.163 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:40.163 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:40.181 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:40.190 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:40.190 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:40.198 INFO:teuthology.orchestra.run.smithi170.stdout:3180 punch from 0x3d56e00 to 0x3d60200, (0x9400 bytes) 2024-09-21T23:16:40.200 INFO:teuthology.orchestra.run.smithi170.stdout:3181 read 0x8f3a00 thru 0x8f8c8a (0x528b bytes) 2024-09-21T23:16:40.203 INFO:teuthology.orchestra.run.smithi170.stdout:3183 read 0x3792400 thru 0x379ddb4 (0xb9b5 bytes) 2024-09-21T23:16:40.204 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:40.203+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:40.204 INFO:teuthology.orchestra.run.smithi170.stdout:3184 read 0xa04200 thru 0xa0470d (0x50e bytes) 2024-09-21T23:16:40.210 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:40.210 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:40.215 INFO:teuthology.orchestra.run.smithi170.stdout:3185 trunc from 0x3eb3400 to 0x8283200 2024-09-21T23:16:40.230 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:40.230 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:40.249 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:40.250 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:40.259 INFO:teuthology.orchestra.run.smithi170.stdout:3186 trunc from 0x8283200 to 0x58a400 2024-09-21T23:16:40.268 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:40.269 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:40.301 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:40.301 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:40.305 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:40.300+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:40.343 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:40.343 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:40.343 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:40.343 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:40.366 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:40.398 INFO:teuthology.orchestra.run.smithi170.stdout:3187 trunc from 0x58a400 to 0x8ea6400 2024-09-21T23:16:40.406 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:40.403+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:40.413 INFO:teuthology.orchestra.run.smithi170.stdout:3188 read 0x5ef4600 thru 0x5f041de (0xfbdf bytes) 2024-09-21T23:16:40.429 INFO:teuthology.orchestra.run.smithi170.stdout:3189 trunc from 0x8ea6400 to 0x41fb200 2024-09-21T23:16:40.448 INFO:teuthology.orchestra.run.smithi170.stdout:3190 punch from 0x135b400 to 0x135de00, (0x2a00 bytes) 2024-09-21T23:16:40.451 INFO:teuthology.orchestra.run.smithi170.stdout:3191 read 0x3526000 thru 0x3529b1d (0x3b1e bytes) 2024-09-21T23:16:40.455 INFO:teuthology.orchestra.run.smithi170.stdout:3192 trunc from 0x41fb200 to 0x2838000 2024-09-21T23:16:40.482 INFO:teuthology.orchestra.run.smithi170.stdout:3193 read 0xa45000 thru 0xa49483 (0x4484 bytes) 2024-09-21T23:16:40.506 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:40.504+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:40.507 INFO:teuthology.orchestra.run.smithi170.stdout:3195 write 0x2e10000 thru 0x2e1cecf (0xced0 bytes) 2024-09-21T23:16:40.507 INFO:teuthology.orchestra.run.smithi170.stdout:3196 read 0xc43800 thru 0xc4e684 (0xae85 bytes) 2024-09-21T23:16:40.507 INFO:teuthology.orchestra.run.smithi170.stdout:3199 read 0x1ff6000 thru 0x20035c9 (0xd5ca bytes) 2024-09-21T23:16:40.532 INFO:teuthology.orchestra.run.smithi170.stdout:3200 trunc from 0x2e1d000 to 0xedb6e00 2024-09-21T23:16:40.533 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:40.533 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:40.533 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:40.533 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:40.533 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:40.548 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:40.559 INFO:teuthology.orchestra.run.smithi170.stdout:3204 punch from 0x9d0f600 to 0x9d17e00, (0x8800 bytes) 2024-09-21T23:16:40.562 INFO:teuthology.orchestra.run.smithi170.stdout:3205 clone 21 order 23 su 65536 sc 7 2024-09-21T23:16:40.634 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:40.632+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:40.709 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:40.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.751365629999995, 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.710 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:16:40.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:16:40.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.751098646000003, 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:40.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:16:40.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138770100999999, 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:16:40.722 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:16:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138727402000001, 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138688781999999, 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:16:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138656910000002, 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.736 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:16:40.740 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138623685999999, 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:16:40.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138588610999999, 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:16:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:16:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138543652999999, 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:16:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.138091586000002, 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:16:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.137377106999999, 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:16:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.137336037000001, 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:16:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.136878919000001, 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:16:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.13664224, 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:16:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.136226658999998, 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:16:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.135548999000001, 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:16:40.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.135057710000002, 2024-09-21T23:16:40.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:16:40.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:16:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.134800812000002, 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.792 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:16:40.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26034 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7325696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:40.796 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.134337730999999, 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086107304999999995, 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26034 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571018+0000", 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018324500000000001 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571023+0000", 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9009999999999999e-06 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571048+0000", 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4454e-05 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571058+0000", 2024-09-21T23:16:40.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0145e-05 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656654+0000", 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085595579000000005 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656758+0000", 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010446999999999999 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656942+0000", 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183511 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26035 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6836224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.134097346000001, 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086157396999999997, 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:40.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26035 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:40.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571805+0000", 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000730326 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571812+0000", 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571833+0000", 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1506999999999999e-05 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571846+0000", 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2330999999999999e-05 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657009+0000", 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085162977000000001 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657091+0000", 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2514000000000003e-05 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657232+0000", 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140858 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:40.806 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:40.704+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:40.807 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:40.809 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:40.811+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:40.910 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:40.908+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:40.976 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:40.976 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:40.989 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:41.034 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:41.032+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:41.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:16:41.119 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:41.119+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:41.159 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:41.159 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:41.159 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:41.159 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:41.160 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:41.173 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:41.218 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:41.216+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:41.296 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:41.296 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:41.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:41.296 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:41.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.296 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:41.296 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.171187494000002, 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:16:41.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:16:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:16:41.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10124 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6156288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:41.302 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166829856, 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038330863999999999, 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10124 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:41.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0399999999999995e-07 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125818+0000", 2024-09-21T23:16:41.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125827+0000", 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6330000000000002e-06 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125834+0000", 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9199999999999998e-06 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.163896+0000", 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038062072000000002 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164009+0000", 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011386400000000001 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164145+0000", 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013601100000000001 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10125 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6141952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166791486000001, 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038589275999999999, 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.307 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10125 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125854+0000", 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125856+0000", 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8159999999999998e-06 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125865+0000", 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7739999999999993e-06 2024-09-21T23:16:41.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125872+0000", 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164189+0000", 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038317276999999997 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164304+0000", 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114676 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164442+0000", 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013814 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10126 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6172672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:41.311 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166763703000001, 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038865093000000003, 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10126 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125884+0000", 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125893+0000", 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125900+0000", 2024-09-21T23:16:41.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0500000000000003e-06 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164491+0000", 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038591473000000001 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164606+0000", 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115109 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164746+0000", 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013963800000000001 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166736337, 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.316 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:41.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:16:41.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166708864, 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:16:41.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166668691000002, 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:16:41.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:16:41.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166617922, 2024-09-21T23:16:41.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:41.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.333 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166587338999999, 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:16:41.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:16:41.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166559433, 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.338 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:16:41.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166532213, 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:16:41.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166505294, 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.347 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:16:41.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166449531000001, 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:16:41.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:16:41.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166416357999999, 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:16:41.356 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:16:41.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.360 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166371138999999, 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.361 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:16:41.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166331859, 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.365 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.369 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166301373, 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166272079999999, 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.374 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:16:41.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:16:41.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.166229737999998, 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:16:41.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:16:41.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.165846543000001, 2024-09-21T23:16:41.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:41.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:16:41.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.387 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.388 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:41.388 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:41.388 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:41.389 INFO:teuthology.orchestra.run.smithi170.stdout:flattening image image_client.0-clone20 2024-09-21T23:16:41.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:41.390 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:41.387+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:41.491 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:41.488+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:41.516 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:41.516 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:41.516 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:41.516 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:41.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:41.592 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:41.588+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:41.658 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:41.658 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:41.658 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:41.658 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:41.658 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:41.672 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:41.719 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:41.719+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:41.793 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:41.788+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.642830977000003, 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:16:41.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:16:41.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.577576219000001, 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:41.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.180234086999999, 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:16:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:16:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:16:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.176866398000001, 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:16:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.172756818, 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:16:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:16:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.168963897000001, 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:16:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.108805090000001, 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:16:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.105048019000002, 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:16:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.105021218000001, 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:16:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104995053, 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:16:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:16:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:16:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104966603000001, 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:16:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:16:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104875894999999, 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:16:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:16:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:16:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104844316000001, 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104806625999998, 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:16:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104776235999999, 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:16:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:16:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104732818999999, 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:16:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.104698194000001, 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:16:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.101444694000001, 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:16:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:16:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:16:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:16:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:16:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:16:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:16:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.101410112, 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.885665713000002, 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:16:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:16:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:41.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:41.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:41.903 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:41.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:41.905 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:41.900+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:42.006 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:42.000+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:42.060 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:16:42.085 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:42.086 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:42.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:42.148 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:42.147+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:42.207 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:42.203+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:42.269 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:42.269 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:42.269 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:42.270 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:42.270 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:42.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:42.304 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-21T23:16:42.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:42.331 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:42.331+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:42.408 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:42.404+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:42.485 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725678107, 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.486 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:42.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.72563285, 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:42.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725583228000001, 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:16:42.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:42.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725553532999999, 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:16:42.502 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:42.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725520605, 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.507 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:42.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725486309000001, 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725461803000002, 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.517 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:42.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:42.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:16:42.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725433812999999, 2024-09-21T23:16:42.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:42.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:16:42.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725410785000001, 2024-09-21T23:16:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:42.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725389275000001, 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:42.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:42.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725353733999999, 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:42.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:42.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:16:42.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725326097, 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:42.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:42.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725290933, 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:42.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:42.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.550 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725255902000001, 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:42.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:42.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.555 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725220517, 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:16:42.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725180335000001, 2024-09-21T23:16:42.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725143206999999, 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:42.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725103132000001, 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725078587999999, 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:42.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.725040802999999, 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:42.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:42.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:42.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:42.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:42.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:42.585 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:42.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:42.587 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:42.587 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:42.587 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:42.587 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:42.588 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:42.588 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:42.588 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:42.589 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:42.583+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:42.593 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:42.593 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:42.611 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:42.611 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:42.687 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:42.684+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:42.722 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:42.722 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:42.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:42.787 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:42.784+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:42.886 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:42.888 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:42.884+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:42.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:42.989 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:42.987+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:43.049 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:43.050 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:43.066 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:43.110 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:43.108+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:43.190 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:43.188+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:43.208 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:43.208 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:43.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:43.310 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:43.308+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:43.375 INFO:teuthology.orchestra.run.smithi170.stdout:3206 read 0x525a200 thru 0x5263a83 (0x9884 bytes) 2024-09-21T23:16:43.377 INFO:teuthology.orchestra.run.smithi170.stdout:3209 read 0x96e3600 thru 0x96e61f8 (0x2bf9 bytes) 2024-09-21T23:16:43.377 INFO:teuthology.orchestra.run.smithi170.stdout:3210 read 0xd49f000 thru 0xd4a90bc (0xa0bd bytes) 2024-09-21T23:16:43.386 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:43.386 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:43.391 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:43.388+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:43.400 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:43.408 INFO:teuthology.orchestra.run.smithi170.stdout:3213 trunc from 0xedb6e00 to 0x9533c00 2024-09-21T23:16:43.452 INFO:teuthology.orchestra.run.smithi170.stdout:3215 punch from 0x6f24200 to 0x6f29800, (0x5600 bytes) 2024-09-21T23:16:43.454 INFO:teuthology.orchestra.run.smithi170.stdout:3217 read 0x5174a00 thru 0x5176199 (0x179a bytes) 2024-09-21T23:16:43.455 INFO:teuthology.orchestra.run.smithi170.stdout:3218 read 0x5385400 thru 0x5390be0 (0xb7e1 bytes) 2024-09-21T23:16:43.480 INFO:teuthology.orchestra.run.smithi170.stdout:3219 write 0xeaf8a00 thru 0xeaf8bad (0x1ae bytes) 2024-09-21T23:16:43.482 INFO:teuthology.orchestra.run.smithi170.stdout:3220 read 0xcc04e00 thru 0xcc122f2 (0xd4f3 bytes) 2024-09-21T23:16:43.482 INFO:teuthology.orchestra.run.smithi170.stdout:3221 read 0x7ccaa00 thru 0x7cd3a31 (0x9032 bytes) 2024-09-21T23:16:43.486 INFO:teuthology.orchestra.run.smithi170.stdout:3224 read 0xd298e00 thru 0xd2a881e (0xfa1f bytes) 2024-09-21T23:16:43.486 INFO:teuthology.orchestra.run.smithi170.stdout:3225 punch from 0x8928800 to 0x892a200, (0x1a00 bytes) 2024-09-21T23:16:43.487 INFO:teuthology.orchestra.run.smithi170.stdout:3226 punch from 0x3690c00 to 0x369f200, (0xe600 bytes) 2024-09-21T23:16:43.487 INFO:teuthology.orchestra.run.smithi170.stdout:3227 read 0x432a800 thru 0x4336cc1 (0xc4c2 bytes) 2024-09-21T23:16:43.488 INFO:teuthology.orchestra.run.smithi170.stdout:3232 read 0x5d21c00 thru 0x5d2a10f (0x8510 bytes) 2024-09-21T23:16:43.488 INFO:teuthology.orchestra.run.smithi170.stdout:3233 read 0x63d1c00 thru 0x63d647a (0x487b bytes) 2024-09-21T23:16:43.488 INFO:teuthology.orchestra.run.smithi170.stdout:3235 punch from 0x7353e00 to 0x7363000, (0xf200 bytes) 2024-09-21T23:16:43.488 INFO:teuthology.orchestra.run.smithi170.stdout:3236 write 0x3bf4e00 thru 0x3bf6a96 (0x1c97 bytes) 2024-09-21T23:16:43.492 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:43.488+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:43.512 INFO:teuthology.orchestra.run.smithi170.stdout:3237 trunc from 0xeaf8c00 to 0x6fe5a00 2024-09-21T23:16:43.554 INFO:teuthology.orchestra.run.smithi170.stdout:3239 read 0x2b99000 thru 0x2b9c3c3 (0x33c4 bytes) 2024-09-21T23:16:43.570 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:43.570 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:43.583 INFO:teuthology.orchestra.run.smithi170.stdout:3241 write 0x7a65800 thru 0x7a73225 (0xda26 bytes) 2024-09-21T23:16:43.586 INFO:teuthology.orchestra.run.smithi170.stdout:3244 punch from 0x67e6a00 to 0x67eca00, (0x6000 bytes) 2024-09-21T23:16:43.587 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:43.588 INFO:teuthology.orchestra.run.smithi170.stdout:3245 write 0x576c400 thru 0x576f593 (0x3194 bytes) 2024-09-21T23:16:43.588 INFO:teuthology.orchestra.run.smithi170.stdout:3247 punch from 0x3a43e00 to 0x3a44600, (0x800 bytes) 2024-09-21T23:16:43.589 INFO:teuthology.orchestra.run.smithi170.stdout:3250 punch from 0x57bca00 to 0x57bd600, (0xc00 bytes) 2024-09-21T23:16:43.598 INFO:teuthology.orchestra.run.smithi170.stdout:3251 trunc from 0x7a73400 to 0xc0aba00 2024-09-21T23:16:43.616 INFO:teuthology.orchestra.run.smithi170.stdout:3259 write 0x99c5600 thru 0x99cdedc (0x88dd bytes) 2024-09-21T23:16:43.618 INFO:teuthology.orchestra.run.smithi170.stdout:3261 read 0x9bf4400 thru 0x9bff4b9 (0xb0ba bytes) 2024-09-21T23:16:43.620 INFO:teuthology.orchestra.run.smithi170.stdout:3264 trunc from 0xc0aba00 to 0xd342600 2024-09-21T23:16:43.630 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:43.628+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:43.655 INFO:teuthology.orchestra.run.smithi170.stdout:3266 trunc from 0xd342600 to 0x8715000 2024-09-21T23:16:43.677 INFO:teuthology.orchestra.run.smithi170.stdout:3268 punch from 0x167a000 to 0x1687200, (0xd200 bytes) 2024-09-21T23:16:43.680 INFO:teuthology.orchestra.run.smithi170.stdout:3271 punch from 0x10e1e00 to 0x10f0800, (0xea00 bytes) 2024-09-21T23:16:43.693 INFO:teuthology.orchestra.run.smithi170.stdout:3272 trunc from 0x8715000 to 0xd1aec00 2024-09-21T23:16:43.694 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:43.692+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:43.714 INFO:teuthology.orchestra.run.smithi170.stdout:3276 read 0x31c0c00 thru 0x31cc6ab (0xbaac bytes) 2024-09-21T23:16:43.717 INFO:teuthology.orchestra.run.smithi170.stdout:3278 write 0xa22f400 thru 0xa239a59 (0xa65a bytes) 2024-09-21T23:16:43.719 INFO:teuthology.orchestra.run.smithi170.stdout:3279 read 0x3d6c00 thru 0x3db083 (0x4484 bytes) 2024-09-21T23:16:43.720 INFO:teuthology.orchestra.run.smithi170.stdout:3280 read 0xcdf6c00 thru 0xce02099 (0xb49a bytes) 2024-09-21T23:16:43.740 INFO:teuthology.orchestra.run.smithi170.stdout:3281 trunc from 0xd1aec00 to 0x6c3e800 2024-09-21T23:16:43.761 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:43.761 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:43.779 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:43.809 INFO:teuthology.orchestra.run.smithi170.stdout:3282 write 0x72df000 thru 0x72ea78f (0xb790 bytes) 2024-09-21T23:16:43.826 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:43.824+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:43.855 INFO:teuthology.orchestra.run.smithi170.stdout:3283 write 0xe8fe400 thru 0xe8fe85d (0x45e bytes) 2024-09-21T23:16:43.856 INFO:teuthology.orchestra.run.smithi170.stdout:3284 read 0xbc45000 thru 0xbc4fb40 (0xab41 bytes) 2024-09-21T23:16:43.856 INFO:teuthology.orchestra.run.smithi170.stdout:3285 write 0x9486e00 thru 0x94886da (0x18db bytes) 2024-09-21T23:16:43.857 INFO:teuthology.orchestra.run.smithi170.stdout:3287 read 0x358c600 thru 0x3591bd5 (0x55d6 bytes) 2024-09-21T23:16:43.875 INFO:teuthology.orchestra.run.smithi170.stdout:3288 trunc from 0xe8fea00 to 0x8983400 2024-09-21T23:16:43.895 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:43.891+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:43.911 INFO:teuthology.orchestra.run.smithi170.stdout:3289 write 0x86b9400 thru 0x86bc6ed (0x32ee bytes) 2024-09-21T23:16:43.912 INFO:teuthology.orchestra.run.smithi170.stdout:3290 clone 22 order 18 su 8192 sc 2 2024-09-21T23:16:43.956 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:43.956 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:43.970 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:43.995 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:43.995+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:44.071 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 1] out_osds: [3] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:16:44.071 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:16:44.071 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T23:16:44.071 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-21T23:16:44.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:16:44.093 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:44.093 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:44.098 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:44.096+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:44.109 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:44.197 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:44.195+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:44.222 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone21 from 0x8983400 (overlap 0x6c3e800) to 0xad3000 2024-09-21T23:16:44.238 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:16:44.238 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:16:44.238 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:16:44.238 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:16:44.238 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:16:44.238 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:16:44.239 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:16:44.239 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:16:44.239 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:16:44.239 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:16:44.239 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:16:44.239 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:16:44.241 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:16:44.242 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:16:44.242 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-21T23:16:44.291 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:44.291 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:44.306 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:44.327 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:44.327+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:44.399 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:44.395+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:44.431 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:44.431 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:16:44.436 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:16:44.440 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:16:44.441 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:16:44.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-21T23:16:44.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:44.527 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:44.527+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:44.572 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:44.572 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:44.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:44.601 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:44.596+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:44.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:44.667 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:16:44.668 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:16:44.672 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:16:44.672 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:16:44.672 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-21T23:16:44.715 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:44.715+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:44.727 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:44.727 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:44.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:44.773 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-21T23:16:44.780 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:44.781 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:44.801 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:44.801 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:44.802 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:44.799+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:44.818 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:44.840 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:44.861 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:44.861 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:44.881 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:16:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:16:44.891 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:16:44.893 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:16:44.893 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:16:44.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-21T23:16:44.903 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:44.900+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:44.924 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:44.924 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:44.924 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:44.924 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:44.925 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:44.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:45.031 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:45.031+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:45.076 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:16:45.076 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:16:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:16:45.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:16:45.079 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:16:45.090 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:16:45.104 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:45.100+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:45.105 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:45.105 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:45.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:45.105 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:45.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.105 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.944984423000001, 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:16:45.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:16:45.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.879729664999999, 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:16:45.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.482387533000001, 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:16:45.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:16:45.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.479019844, 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:16:45.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:16:45.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.474910264000002, 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:16:45.125 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:16:45.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.471117343000003, 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:16:45.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:16:45.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.410958536000003, 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.134 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:45.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.407201465, 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:16:45.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:16:45.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.407174664000003, 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:16:45.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.407148499000002, 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:16:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.407120049, 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:16:45.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:16:45.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.407029340999998, 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:16:45.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.406997762000003, 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:16:45.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.406960071999997, 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:16:45.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:16:45.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.406929681999998, 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:45.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.406886264999997, 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:16:45.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.406851639999999, 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:16:45.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:16:45.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.40359814, 2024-09-21T23:16:45.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:16:45.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.403563558000002, 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:16:45.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:16:45.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.187819159, 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:16:45.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:16:45.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:16:45.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.198 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:45.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:45.244 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:45.243+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:45.305 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:45.300+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5538.0:1159 3.4 3:2161c908:::rbd_data.10f1d740b9a0.000000000000006c:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:45.378405+0000", 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00035963400000000001, 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040798400000000002, 2024-09-21T23:16:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5538", 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/251089975", 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1159 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378405+0000", 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378405+0000", 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378406+0000", 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378411+0000", 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8330000000000001e-06 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378424+0000", 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378432+0000", 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0069999999999997e-06 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378584+0000", 2024-09-21T23:16:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015202099999999999 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5538.0:1160 3.4 3.10938684 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:45.378555+0000", 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.000210053, 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041373000000000001, 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5538", 2024-09-21T23:16:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/251089975", 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1160 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378555+0000", 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378555+0000", 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378556+0000", 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-06 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378563+0000", 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2169999999999997e-06 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378586+0000", 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2517000000000001e-05 2024-09-21T23:16:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378604+0000", 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8209000000000001e-05 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:45.378940+0000", 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033567500000000001 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:16:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 2 2024-09-21T23:16:45.384 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:45.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:45.439 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:45.439+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:45.507 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:45.503+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:45.515 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:45.515 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:45.515 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:45.515 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:45.515 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:45.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:45.615 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:45.615+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940380732000001, 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:45.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940335475000001, 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.708 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:45.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:16:45.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940285852999999, 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:45.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940256158, 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:45.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:45.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940223230000001, 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:45.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940188933999998, 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.728 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:45.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.732 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940164427999999, 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940136438, 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:45.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:16:45.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940113409999999, 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:45.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:45.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940091899999999, 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:45.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:45.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940056359, 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:45.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.940028722000001, 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:45.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:16:45.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939993558000001, 2024-09-21T23:16:45.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:16:45.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939958527000002, 2024-09-21T23:16:45.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:45.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:16:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939923142000001, 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:45.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939882959999998, 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:45.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939845832, 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:45.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:45.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:16:45.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939805756999998, 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939781213, 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:45.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:45.795 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.939743428, 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:45.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:45.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:45.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:45.800 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:45.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:45.802 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:45.799+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:45.902 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:45.900+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:45.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:46.003 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:46.000+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:46.104 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:46.100+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:46.128 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:46.128 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:46.128 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:46.128 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:46.128 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:46.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:46.205 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:46.200+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:46.266 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:46.266 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:46.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:46.266 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.338149650999995, 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:46.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:46.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:46.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.337128804000002, 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:46.272 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:46.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.337069131000007, 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336979154999995, 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:46.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:46.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:46.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336929244999993, 2024-09-21T23:16:46.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:46.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:46.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336873104999995, 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.293 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:46.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:46.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336819321999997, 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:46.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336787045999998, 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.303 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:46.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:46.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336756765999993, 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:46.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:46.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336727109999998, 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.313 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:46.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.318 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336697142999995, 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:46.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336619314000004, 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:46.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:46.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:46.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:46.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336571606999996, 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:46.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:46.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336541410999999, 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:46.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336511494000007, 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:46.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:46.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336481694, 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:46.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:46.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336451761000006, 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:46.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:46.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:46.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336386798000007, 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:46.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336356322, 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:46.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.336327158000003, 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:46.372 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:46.373 INFO:teuthology.orchestra.run.smithi170.stdout:3291 punch from 0x7aa7400 to 0x7ab3200, (0xbe00 bytes) 2024-09-21T23:16:46.374 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:46.374 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:46.371+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:46.391 INFO:teuthology.orchestra.run.smithi170.stdout:3295 write 0x2e56400 thru 0x2e61dea (0xb9eb bytes) 2024-09-21T23:16:46.409 INFO:teuthology.orchestra.run.smithi170.stdout:3297 trunc from 0x8983400 to 0xb7e6000 2024-09-21T23:16:46.452 INFO:teuthology.orchestra.run.smithi170.stdout:3300 write 0x1029e00 thru 0x102cba3 (0x2da4 bytes) 2024-09-21T23:16:46.453 INFO:teuthology.orchestra.run.smithi170.stdout:3301 read 0xab99e00 thru 0xab9e0ef (0x42f0 bytes) 2024-09-21T23:16:46.453 INFO:teuthology.orchestra.run.smithi170.stdout:3302 punch from 0x5102600 to 0x510ee00, (0xc800 bytes) 2024-09-21T23:16:46.455 INFO:teuthology.orchestra.run.smithi170.stdout:3303 punch from 0xa3ef600 to 0xa3f1200, (0x1c00 bytes) 2024-09-21T23:16:46.455 INFO:teuthology.orchestra.run.smithi170.stdout:3304 write 0xb441400 thru 0xb44e256 (0xce57 bytes) 2024-09-21T23:16:46.455 INFO:teuthology.orchestra.run.smithi170.stdout:3306 write 0x1322400 thru 0x132f550 (0xd151 bytes) 2024-09-21T23:16:46.459 INFO:teuthology.orchestra.run.smithi170.stdout:3307 read 0x2723c00 thru 0x272b26f (0x7670 bytes) 2024-09-21T23:16:46.461 INFO:teuthology.orchestra.run.smithi170.stdout:3308 punch from 0x92ba200 to 0x92c7200, (0xd000 bytes) 2024-09-21T23:16:46.463 INFO:teuthology.orchestra.run.smithi170.stdout:3310 flatten 2024-09-21T23:16:46.474 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:46.472+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:46.519 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:46.519 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:46.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:46.578 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:46.576+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:46.676 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:46.672+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:46.705 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:46.706 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:46.706 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:46.706 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:46.706 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:46.721 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:46.776 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:46.775+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:46.880 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:46.876+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:46.894 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:46.894 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:46.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:46.894 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.672547365, 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.895 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:16:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:16:46.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:16:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.16014208199999, 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:46.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:16:46.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:16:46.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.160077179, 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:16:46.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.15993073999999, 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:16:46.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:46.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:16:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:16:46.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:46.919 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.159861625, 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:46.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.1592493, 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:16:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:16:46.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:16:46.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:16:46.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.15904489499999, 2024-09-21T23:16:46.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:16:46.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 120.159002109, 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.937 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.942 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.76804382100001, 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:16:46.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:16:46.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:16:46.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:16:46.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:16:46.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.767957311, 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.948 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:46.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:16:46.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:16:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:16:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.767880422, 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:16:46.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:16:46.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.959 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.767675582, 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:16:46.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:16:46.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.965 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.7675873, 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.76745647200001, 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.971 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:16:46.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:16:46.976 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.76741081900001, 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:16:46.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:16:46.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.767372392, 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:16:46.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:16:46.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.988 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.76724292999999, 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:46.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.994 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.767221288, 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:46.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:16:46.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:16:46.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:46.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:16:47.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:16:47.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:47.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:16:47.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:16:47.001 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.001 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.76720124400001, 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:47.002 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.007 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.76711725200001, 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:47.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:47.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:47.013 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:47.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:47.015 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:47.015 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:47.012+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:47.115 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:47.112+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:47.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:16:47.202 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:47.202 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:47.230 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:47.230 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:47.231 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:47.227+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5320.0:13576 3.11 3.f7191cb1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:47.239343+0000", 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 1.8050000000000002e-05, 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5626e-05, 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5320", 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2539639432", 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 13576 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239343+0000", 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239343+0000", 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239343+0000", 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6199999999999997e-07 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239347+0000", 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1669999999999999e-06 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239357+0000", 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9240000000000002e-06 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239363+0000", 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5690000000000001e-06 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5320.0:13577 3.11 3.f7191cb1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-21T23:16:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:47.239381+0000", 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9999804, 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2284999999999998e-05, 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5320", 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2539639432", 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 13577 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239381+0000", 2024-09-21T23:16:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239381+0000", 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239381+0000", 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239383+0000", 2024-09-21T23:16:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2749999999999998e-06 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239392+0000", 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9109999999999999e-06 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:47.239398+0000", 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6540000000000004e-06 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:16:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 2 2024-09-21T23:16:47.247 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:47.254 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:47.254 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:47.260 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:47.282 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:47.282 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:47.316 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:47.317 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:47.317 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:47.315+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:47.345 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:47.346 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:47.353 INFO:teuthology.orchestra.run.smithi170.stdout:3311 write 0x5338800 thru 0x533a9b6 (0x21b7 bytes) 2024-09-21T23:16:47.393 INFO:teuthology.orchestra.run.smithi170.stdout:3314 trunc from 0xb7e6000 to 0xe5ed800 2024-09-21T23:16:47.412 INFO:teuthology.orchestra.run.smithi170.stdout:3315 read 0x26b400 thru 0x26e877 (0x3478 bytes) 2024-09-21T23:16:47.415 INFO:teuthology.orchestra.run.smithi170.stdout:3316 punch from 0xf37e00 to 0xf39e00, (0x2000 bytes) 2024-09-21T23:16:47.415 INFO:teuthology.orchestra.run.smithi170.stdout:3317 write 0x937a00 thru 0x93cbb8 (0x51b9 bytes) 2024-09-21T23:16:47.416 INFO:teuthology.orchestra.run.smithi170.stdout:3319 write 0x7266c00 thru 0x726d37b (0x677c bytes) 2024-09-21T23:16:47.416 INFO:teuthology.orchestra.run.smithi170.stdout:3320 write 0xb73fe00 thru 0xb746086 (0x6287 bytes) 2024-09-21T23:16:47.417 INFO:teuthology.orchestra.run.smithi170.stdout:3321 write 0x44cee00 thru 0x44d8791 (0x9992 bytes) 2024-09-21T23:16:47.417 INFO:teuthology.orchestra.run.smithi170.stdout:3325 read 0xb7b800 thru 0xb8055b (0x4d5c bytes) 2024-09-21T23:16:47.417 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:47.415+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:47.418 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:47.418 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:47.430 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:47.453 INFO:teuthology.orchestra.run.smithi170.stdout:3326 trunc from 0xe5ed800 to 0x46ebc00 2024-09-21T23:16:47.517 INFO:teuthology.orchestra.run.smithi170.stdout:3327 read 0x2507800 thru 0x25176a6 (0xfea7 bytes) 2024-09-21T23:16:47.518 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:47.515+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:47.519 INFO:teuthology.orchestra.run.smithi170.stdout:3328 read 0x1eedc00 thru 0x1efb952 (0xdd53 bytes) 2024-09-21T23:16:47.519 INFO:teuthology.orchestra.run.smithi170.stdout:3330 read 0x2843a00 thru 0x2844fa6 (0x15a7 bytes) 2024-09-21T23:16:47.519 INFO:teuthology.orchestra.run.smithi170.stdout:skipping zero length punch hole 2024-09-21T23:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout:3338 write 0x635fa00 thru 0x636ebb2 (0xf1b3 bytes) 2024-09-21T23:16:47.554 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:47.554 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:47.568 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:47.570 INFO:teuthology.orchestra.run.smithi170.stdout:3341 write 0x6c50200 thru 0x6c5f88a (0xf68b bytes) 2024-09-21T23:16:47.573 INFO:teuthology.orchestra.run.smithi170.stdout:3343 read 0xed8e00 thru 0xeda8a5 (0x1aa6 bytes) 2024-09-21T23:16:47.573 INFO:teuthology.orchestra.run.smithi170.stdout:3345 read 0x2532800 thru 0x25379d0 (0x51d1 bytes) 2024-09-21T23:16:47.619 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:47.615+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:47.622 INFO:teuthology.orchestra.run.smithi170.stdout:3346 trunc from 0x6c5fa00 to 0x1522000 2024-09-21T23:16:47.690 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:47.690 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:47.703 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:47.723 INFO:teuthology.orchestra.run.smithi170.stdout:3347 write 0xd8d2a00 thru 0xd8d7880 (0x4e81 bytes) 2024-09-21T23:16:47.727 INFO:teuthology.orchestra.run.smithi170.stdout:3348 read 0x5414c00 thru 0x5417e5e (0x325f bytes) 2024-09-21T23:16:47.728 INFO:teuthology.orchestra.run.smithi170.stdout:3349 read 0xc92ba00 thru 0xc9388b6 (0xceb7 bytes) 2024-09-21T23:16:47.728 INFO:teuthology.orchestra.run.smithi170.stdout:3351 clone 23 order 23 su 2097152 sc 12 2024-09-21T23:16:47.750 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:47.748+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:47.820 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:47.820 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:47.820 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:47.819+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:47.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:47.922 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:47.923+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:47.993 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:47.993 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:48.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:48.023 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:48.020+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:16:48.123 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:48.120+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:48.151 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:48.151 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:48.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:48.251 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:48.251+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:48.328 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:48.327+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:48.330 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:48.330 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:48.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:48.431 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:48.431+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:48.521 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:48.571 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:48.573 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:48.571+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:48.573 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:48.674 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:48.671+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:48.750 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:48.750 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:48.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:48.807 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:48.807+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:48.853 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-21T23:16:48.876 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:48.872+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:48.948 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:48.948 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:48.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:49.007 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:49.007+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:49.016 INFO:teuthology.orchestra.run.smithi170.stdout:3352 read 0x4172c00 thru 0x4179c8a (0x708b bytes) 2024-09-21T23:16:49.017 INFO:teuthology.orchestra.run.smithi170.stdout:3354 read 0x2a28e00 thru 0x2a345fc (0xb7fd bytes) 2024-09-21T23:16:49.018 INFO:teuthology.orchestra.run.smithi170.stdout:3356 write 0xad5a000 thru 0xad5c67b (0x267c bytes) 2024-09-21T23:16:49.018 INFO:teuthology.orchestra.run.smithi170.stdout:3357 write 0xbf99a00 thru 0xbf9fe84 (0x6485 bytes) 2024-09-21T23:16:49.019 INFO:teuthology.orchestra.run.smithi170.stdout:3358 read 0x5582a00 thru 0x558db86 (0xb187 bytes) 2024-09-21T23:16:49.020 INFO:teuthology.orchestra.run.smithi170.stdout:3359 punch from 0xd3c7800 to 0xd3d4800, (0xd000 bytes) 2024-09-21T23:16:49.039 INFO:teuthology.orchestra.run.smithi170.stdout:3360 read 0x5dae600 thru 0x5db6fc4 (0x89c5 bytes) 2024-09-21T23:16:49.039 INFO:teuthology.orchestra.run.smithi170.stdout:3361 read 0xbd50200 thru 0xbd58122 (0x7f23 bytes) 2024-09-21T23:16:49.039 INFO:teuthology.orchestra.run.smithi170.stdout:3362 read 0xd4b6000 thru 0xd4bb32b (0x532c bytes) 2024-09-21T23:16:49.040 INFO:teuthology.orchestra.run.smithi170.stdout:3368 write 0x1b91600 thru 0x1b936d4 (0x20d5 bytes) 2024-09-21T23:16:49.078 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:49.076+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:49.086 INFO:teuthology.orchestra.run.smithi170.stdout:3372 trunc from 0xd8d7a00 to 0x5814200 2024-09-21T23:16:49.091 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:49.092 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:49.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:49.138 INFO:teuthology.orchestra.run.smithi170.stdout:3373 punch from 0x2c62600 to 0x2c69200, (0x6c00 bytes) 2024-09-21T23:16:49.140 INFO:teuthology.orchestra.run.smithi170.stdout:3374 read 0x8ffa00 thru 0x90efb5 (0xf5b6 bytes) 2024-09-21T23:16:49.141 INFO:teuthology.orchestra.run.smithi170.stdout:3376 read 0x49e8a00 thru 0x49ec4e0 (0x3ae1 bytes) 2024-09-21T23:16:49.147 INFO:teuthology.orchestra.run.smithi170.stdout:3378 trunc from 0x5814200 to 0x8173200 2024-09-21T23:16:49.170 INFO:teuthology.orchestra.run.smithi170.stdout:3379 read 0x1edee00 thru 0x1ee44e8 (0x56e9 bytes) 2024-09-21T23:16:49.171 INFO:teuthology.orchestra.run.smithi170.stdout:3381 read 0x26f8c00 thru 0x27042f6 (0xb6f7 bytes) 2024-09-21T23:16:49.188 INFO:teuthology.orchestra.run.smithi170.stdout:3382 trunc from 0x8173200 to 0xe4e6e00 2024-09-21T23:16:49.199 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:49.199+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:49.209 INFO:teuthology.orchestra.run.smithi170.stdout:3384 read 0x607a000 thru 0x607dbd2 (0x3bd3 bytes) 2024-09-21T23:16:49.265 INFO:teuthology.orchestra.run.smithi170.stdout:3385 trunc from 0xe4e6e00 to 0x2c80800 2024-09-21T23:16:49.279 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:49.275+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:49.281 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:49.282 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:49.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:49.345 INFO:teuthology.orchestra.run.smithi170.stdout:3387 write 0x3007600 thru 0x300dfc2 (0x69c3 bytes) 2024-09-21T23:16:49.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:49.380 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:49.376+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:3391 write 0xae07400 thru 0xae15f7d (0xeb7e bytes) 2024-09-21T23:16:49.403 INFO:teuthology.orchestra.run.smithi170.stdout:3392 read 0x3cd9600 thru 0x3cdf477 (0x5e78 bytes) 2024-09-21T23:16:49.405 INFO:teuthology.orchestra.run.smithi170.stdout:3394 flatten 2024-09-21T23:16:49.437 INFO:teuthology.orchestra.run.smithi170.stdout:3395 read 0x45da400 thru 0x45e3de6 (0x99e7 bytes) 2024-09-21T23:16:49.439 INFO:teuthology.orchestra.run.smithi170.stdout:3396 write 0x80ba600 thru 0x80c5131 (0xab32 bytes) 2024-09-21T23:16:49.442 INFO:teuthology.orchestra.run.smithi170.stdout:3398 trunc from 0xae16000 to 0x9d2ae00 2024-09-21T23:16:49.477 INFO:teuthology.orchestra.run.smithi170.stdout:3399 read 0x42b6000 thru 0x42babfb (0x4bfc bytes) 2024-09-21T23:16:49.479 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:49.479 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:49.480 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:49.476+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:49.481 INFO:teuthology.orchestra.run.smithi170.stdout:3400 write 0x277a400 thru 0x277a600 (0x201 bytes) 2024-09-21T23:16:49.493 INFO:teuthology.orchestra.run.smithi170.stdout:3402 trunc from 0x9d2ae00 to 0x568de00 2024-09-21T23:16:49.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:49.531 INFO:teuthology.orchestra.run.smithi170.stdout:3403 punch from 0x243b200 to 0x2447200, (0xc000 bytes) 2024-09-21T23:16:49.533 INFO:teuthology.orchestra.run.smithi170.stdout:3404 read 0x4679000 thru 0x468047d (0x747e bytes) 2024-09-21T23:16:49.538 INFO:teuthology.orchestra.run.smithi170.stdout:3405 trunc from 0x568de00 to 0x3d58c00 2024-09-21T23:16:49.558 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:49.564 INFO:teuthology.orchestra.run.smithi170.stdout:3406 read 0x2a63c00 thru 0x2a70a7c (0xce7d bytes) 2024-09-21T23:16:49.566 INFO:teuthology.orchestra.run.smithi170.stdout:3407 read 0x19d6800 thru 0x19e41e4 (0xd9e5 bytes) 2024-09-21T23:16:49.568 INFO:teuthology.orchestra.run.smithi170.stdout:3408 read 0x1da5e00 thru 0x1db0e91 (0xb092 bytes) 2024-09-21T23:16:49.581 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:49.579+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:49.585 INFO:teuthology.orchestra.run.smithi170.stdout:3414 trunc from 0x3d58c00 to 0xa070600 2024-09-21T23:16:49.591 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:49.591 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:49.605 INFO:teuthology.orchestra.run.smithi170.stdout:3418 write 0x965c000 thru 0x965d534 (0x1535 bytes) 2024-09-21T23:16:49.606 INFO:teuthology.orchestra.run.smithi170.stdout:3419 write 0x571b800 thru 0x572adab (0xf5ac bytes) 2024-09-21T23:16:49.608 INFO:teuthology.orchestra.run.smithi170.stdout:3424 write 0x64f6c00 thru 0x64fdcf0 (0x70f1 bytes) 2024-09-21T23:16:49.608 INFO:teuthology.orchestra.run.smithi170.stdout:3426 read 0x912b800 thru 0x9130d69 (0x556a bytes) 2024-09-21T23:16:49.608 INFO:teuthology.orchestra.run.smithi170.stdout:3427 trunc from 0xa070600 to 0xa029800 2024-09-21T23:16:49.608 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:49.609 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:49.633 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:49.637 INFO:teuthology.orchestra.run.smithi170.stdout:3428 read 0x36f8a00 thru 0x37076fd (0xecfe bytes) 2024-09-21T23:16:49.646 INFO:teuthology.orchestra.run.smithi170.stdout:3431 trunc from 0xa029800 to 0xce16c00 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.722894116999996, 2024-09-21T23:16:49.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:49.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:49.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721873270000003, 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:49.657 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:49.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.662 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721813596999993, 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:49.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:49.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721723620999995, 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:49.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:49.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721673710999994, 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:49.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:49.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:49.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721617570999996, 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.679 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:49.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721563787999997, 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:49.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:49.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721531511999999, 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:49.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:49.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721501231999994, 2024-09-21T23:16:49.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:49.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:49.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721471575999999, 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:49.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721441608999996, 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:49.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721363780000004, 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:49.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:49.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721316072999997, 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.715 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:49.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.720 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721285877, 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:49.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.725 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721255959999993, 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.730 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721226160000001, 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:49.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721196226999993, 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.736 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:49.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:49.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721131263999993, 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:49.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:49.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721100788000001, 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:49.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:49.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.721071624000004, 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:49.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:49.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:49.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:49.756 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:49.757 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:49.758 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3432 read 0x77f3e00 thru 0x7802cb8 (0xeeb9 bytes) 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3434 punch from 0xc31e600 to 0xc32ae00, (0xc800 bytes) 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3438 trunc from 0xce16c00 to 0x8bdfa00 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3439 read 0x46a9400 thru 0x46b726b (0xde6c bytes) 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3440 punch from 0x3585a00 to 0x3593400, (0xda00 bytes) 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3443 punch from 0x72b3400 to 0x72b9800, (0x6400 bytes) 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3444 read 0x64cd600 thru 0x64cdf25 (0x926 bytes) 2024-09-21T23:16:49.759 INFO:teuthology.orchestra.run.smithi170.stdout:3445 write 0xb732400 thru 0xb739d22 (0x7923 bytes) 2024-09-21T23:16:49.760 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:49.760 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:49.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:49.761 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:49.756+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:49.762 INFO:teuthology.orchestra.run.smithi170.stdout:3447 trunc from 0xb739e00 to 0x75d9200 2024-09-21T23:16:49.787 INFO:teuthology.orchestra.run.smithi170.stdout:3448 punch from 0x1dec00 to 0x1e5600, (0x6a00 bytes) 2024-09-21T23:16:49.859 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:49.855+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:49.894 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:49.895 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:49.895 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:49.895 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:49.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:49.960 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:49.956+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:50.025 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:50.026 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:50.035 INFO:teuthology.orchestra.run.smithi170.stdout:3449 write 0xe91f400 thru 0xe921843 (0x2444 bytes) 2024-09-21T23:16:50.035 INFO:teuthology.orchestra.run.smithi170.stdout:3452 write 0x2e63200 thru 0x2e6dbc8 (0xa9c9 bytes) 2024-09-21T23:16:50.036 INFO:teuthology.orchestra.run.smithi170.stdout:3454 write 0x2d49200 thru 0x2d5043b (0x723c bytes) 2024-09-21T23:16:50.037 INFO:teuthology.orchestra.run.smithi170.stdout:3456 read 0xfdfc00 thru 0xfe2a94 (0x2e95 bytes) 2024-09-21T23:16:50.039 INFO:teuthology.orchestra.run.smithi170.stdout:3460 write 0x5009600 thru 0x501307d (0x9a7e bytes) 2024-09-21T23:16:50.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:50.040 INFO:teuthology.orchestra.run.smithi170.stdout:3463 write 0xb5d200 thru 0xb655bf (0x83c0 bytes) 2024-09-21T23:16:50.052 INFO:teuthology.orchestra.run.smithi170.stdout:3465 trunc from 0xe921a00 to 0xab89400 2024-09-21T23:16:50.062 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:50.059+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:50.080 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 1] out_osds: [3] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:16:50.080 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:16:50.080 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 2, 4, 5, 1] 2024-09-21T23:16:50.080 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-21T23:16:50.092 INFO:teuthology.orchestra.run.smithi170.stdout:3468 read 0x5d6dc00 thru 0x5d790f3 (0xb4f4 bytes) 2024-09-21T23:16:50.101 INFO:teuthology.orchestra.run.smithi170.stdout:3470 trunc from 0xab89400 to 0x8200a00 2024-09-21T23:16:50.118 INFO:teuthology.orchestra.run.smithi170.stdout:3474 punch from 0x558b400 to 0x5599e00, (0xea00 bytes) 2024-09-21T23:16:50.120 INFO:teuthology.orchestra.run.smithi170.stdout:3475 write 0x6f7f000 thru 0x6f8c49b (0xd49c bytes) 2024-09-21T23:16:50.120 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:16:50.154 INFO:teuthology.orchestra.run.smithi170.stdout:3477 write 0xda60e00 thru 0xda62ee2 (0x20e3 bytes) 2024-09-21T23:16:50.154 INFO:teuthology.orchestra.run.smithi170.stdout:3479 read 0x791aa00 thru 0x79291d8 (0xe7d9 bytes) 2024-09-21T23:16:50.161 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:50.156+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:50.186 INFO:teuthology.orchestra.run.smithi170.stdout:3481 trunc from 0xda63000 to 0x39e200 2024-09-21T23:16:50.213 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:50.213 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.990595337, 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:16:50.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:16:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:50.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.220 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.478190054, 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.221 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:16:50.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:16:50.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.478125151, 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.227 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:16:50.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:16:50.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.232 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.477978712, 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:16:50.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.238 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.47790959700001, 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.239 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:16:50.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:50.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.477297272, 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:16:50.245 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:16:50.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:16:50.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:16:50.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.477092867, 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:16:50.251 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.256 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.477050081, 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.257 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:16:50.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:16:50.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08609179299999, 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:16:50.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:16:50.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:50.272 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08600528300001, 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:50.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08592839400001, 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.278 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.085723554, 2024-09-21T23:16:50.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:16:50.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:16:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.085635272, 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:16:50.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08550444399999, 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:16:50.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:16:50.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08545879099999, 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:16:50.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:16:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.085420364, 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:16:50.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:16:50.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.085290902, 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.312 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:16:50.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:16:50.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:16:50.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08526926, 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:16:50.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:16:50.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:16:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:16:50.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08524921599999, 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:16:50.323 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:16:50.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:16:50.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 123.08516522399999, 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.329 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:16:50.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:16:50.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:16:50.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.335 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.335 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.335 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:50.337 INFO:teuthology.orchestra.run.smithi170.stdout:3483 write 0x69a9200 thru 0x69b0011 (0x6e12 bytes) 2024-09-21T23:16:50.337 INFO:teuthology.orchestra.run.smithi170.stdout:3485 write 0x6802400 thru 0x6804160 (0x1d61 bytes) 2024-09-21T23:16:50.337 INFO:teuthology.orchestra.run.smithi170.stdout:3486 read 0x5c0ea00 thru 0x5c0f6ba (0xcbb bytes) 2024-09-21T23:16:50.337 INFO:teuthology.orchestra.run.smithi170.stdout:3488 write 0xc86b200 thru 0xc873564 (0x8365 bytes) 2024-09-21T23:16:50.337 INFO:teuthology.orchestra.run.smithi170.stdout:3489 write 0x721b800 thru 0x7229079 (0xd87a bytes) 2024-09-21T23:16:50.337 INFO:teuthology.orchestra.run.smithi170.stdout:3490 trunc from 0xc873600 to 0xa7d9c00 2024-09-21T23:16:50.337 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:50.338 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:50.335+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:50.374 INFO:teuthology.orchestra.run.smithi170.stdout:3491 trunc from 0xa7d9c00 to 0x2e40600 2024-09-21T23:16:50.404 INFO:teuthology.orchestra.run.smithi170.stdout:3492 punch from 0x2501200 to 0x2508400, (0x7200 bytes) 2024-09-21T23:16:50.428 INFO:teuthology.orchestra.run.smithi170.stdout:3493 write 0x75e2c00 thru 0x75e6721 (0x3b22 bytes) 2024-09-21T23:16:50.429 INFO:teuthology.orchestra.run.smithi170.stdout:3496 read 0x42e1800 thru 0x42e4d49 (0x354a bytes) 2024-09-21T23:16:50.437 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:50.435+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:50.454 INFO:teuthology.orchestra.run.smithi170.stdout:3499 write 0x9ed4800 thru 0x9edaa45 (0x6246 bytes) 2024-09-21T23:16:50.454 INFO:teuthology.orchestra.run.smithi170.stdout:3501 punch from 0x800 to 0xd800, (0xd000 bytes) 2024-09-21T23:16:50.457 INFO:teuthology.orchestra.run.smithi170.stdout:3502 read 0x4f34800 thru 0x4f3d4e7 (0x8ce8 bytes) 2024-09-21T23:16:50.458 INFO:teuthology.orchestra.run.smithi170.stdout:3504 punch from 0x315000 to 0x31d400, (0x8400 bytes) 2024-09-21T23:16:50.486 INFO:teuthology.orchestra.run.smithi170.stdout:3506 write 0xd4f5200 thru 0xd4faecd (0x5cce bytes) 2024-09-21T23:16:50.487 INFO:teuthology.orchestra.run.smithi170.stdout:3507 write 0x50b200 thru 0x513705 (0x8506 bytes) 2024-09-21T23:16:50.495 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-21T23:16:50.495+0000 7faab395a640 -1 mon.a@0(leader).osd e95 definitely_dead 0 2024-09-21T23:16:50.505 INFO:teuthology.orchestra.run.smithi170.stdout:3508 trunc from 0xd4fb000 to 0x4086a00 2024-09-21T23:16:50.516 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:50.516 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:50.532 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:50.537 INFO:teuthology.orchestra.run.smithi170.stdout:3510 punch from 0xa6aa00 to 0xa7a200, (0xf800 bytes) 2024-09-21T23:16:50.542 INFO:teuthology.orchestra.run.smithi170.stdout:3511 trunc from 0x4086a00 to 0x164ce00 2024-09-21T23:16:50.577 INFO:teuthology.orchestra.run.smithi170.stdout:3512 trunc from 0x164ce00 to 0xc501a00 2024-09-21T23:16:50.578 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:50.576+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:3514 read 0xb100c00 thru 0xb10c580 (0xb981 bytes) 2024-09-21T23:16:50.593 INFO:teuthology.orchestra.run.smithi170.stdout:3515 punch from 0x8b74e00 to 0x8b77000, (0x2200 bytes) 2024-09-21T23:16:50.594 INFO:teuthology.orchestra.run.smithi170.stdout:3516 write 0x9622200 thru 0x962c3c1 (0xa1c2 bytes) 2024-09-21T23:16:50.594 INFO:teuthology.orchestra.run.smithi170.stdout:3517 read 0xa4f8600 thru 0xa4fc67c (0x407d bytes) 2024-09-21T23:16:50.594 INFO:teuthology.orchestra.run.smithi170.stdout:3520 read 0x9c40200 thru 0x9c459c3 (0x57c4 bytes) 2024-09-21T23:16:50.594 INFO:teuthology.orchestra.run.smithi170.stdout:3521 punch from 0x8010200 to 0x8014c00, (0x4a00 bytes) 2024-09-21T23:16:50.614 INFO:teuthology.orchestra.run.smithi170.stdout:3525 write 0xd390600 thru 0xd396017 (0x5a18 bytes) 2024-09-21T23:16:50.623 INFO:teuthology.orchestra.run.smithi170.stdout:3526 trunc from 0xd396200 to 0x9174000 2024-09-21T23:16:50.638 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:50.635+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:50.655 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:50.655 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:50.665 INFO:teuthology.orchestra.run.smithi170.stdout:3527 write 0x5869000 thru 0x586f61d (0x661e bytes) 2024-09-21T23:16:50.666 INFO:teuthology.orchestra.run.smithi170.stdout:3529 read 0x6475400 thru 0x64790fc (0x3cfd bytes) 2024-09-21T23:16:50.666 INFO:teuthology.orchestra.run.smithi170.stdout:3530 write 0x2e02400 thru 0x2e0e925 (0xc526 bytes) 2024-09-21T23:16:50.666 INFO:teuthology.orchestra.run.smithi170.stdout:3531 read 0x8313400 thru 0x831f873 (0xc474 bytes) 2024-09-21T23:16:50.668 INFO:teuthology.orchestra.run.smithi170.stdout:3532 trunc from 0x9174000 to 0x822ba00 2024-09-21T23:16:50.670 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:50.701 INFO:teuthology.orchestra.run.smithi170.stdout:3533 write 0x6e9a800 thru 0x6ea8a72 (0xe273 bytes) 2024-09-21T23:16:50.704 INFO:teuthology.orchestra.run.smithi170.stdout:3534 write 0x6b44a00 thru 0x6b518e1 (0xcee2 bytes) 2024-09-21T23:16:50.708 INFO:teuthology.orchestra.run.smithi170.stdout:3536 trunc from 0x822ba00 to 0x992e400 2024-09-21T23:16:50.762 INFO:teuthology.orchestra.run.smithi170.stdout:3538 trunc from 0x992e400 to 0xe83000 2024-09-21T23:16:50.762 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:50.760+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:50.813 INFO:teuthology.orchestra.run.smithi170.stdout:3539 read 0xb7f600 thru 0xb86870 (0x7271 bytes) 2024-09-21T23:16:50.817 INFO:teuthology.orchestra.run.smithi170.stdout:3541 punch from 0x12800 to 0x20a00, (0xe200 bytes) 2024-09-21T23:16:50.817 INFO:teuthology.orchestra.run.smithi170.stdout:3543 read 0x859200 thru 0x867633 (0xe434 bytes) 2024-09-21T23:16:50.817 INFO:teuthology.orchestra.run.smithi170.stdout:3544 punch from 0xec00 to 0x13200, (0x4600 bytes) 2024-09-21T23:16:50.823 INFO:teuthology.orchestra.run.smithi170.stdout:3545 trunc from 0xe83000 to 0xdbee00 2024-09-21T23:16:50.839 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:50.836+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.888017464000001, 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.846 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:16:50.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:16:50.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.852 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.887750479999994, 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.853 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:16:50.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:16:50.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.275421934999997, 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.860 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:16:50.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.275379235999999, 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:16:50.864 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:16:50.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.275340616000001, 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:16:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.275308744, 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:16:50.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.275275520000001, 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.878 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:16:50.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:16:50.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.275240445000001, 2024-09-21T23:16:50.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:50.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:16:50.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.275195486999998, 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:16:50.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.27474342, 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:16:50.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:16:50.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.274028940999997, 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:16:50.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:50.900 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.273987871000003, 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:50.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.904 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.273530753000003, 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.905 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:16:50.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.273294073999999, 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.909 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:16:50.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.272878493, 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:16:50.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:16:50.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.272200832999999, 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:16:50.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:16:50.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:50.922 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.271709543999997, 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:50.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:16:50.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.926 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.271452646, 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.927 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:16:50.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26034 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7325696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:50.931 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.270989565000001, 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086107304999999995, 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26034 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:50.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571018+0000", 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018324500000000001 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571023+0000", 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9009999999999999e-06 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571048+0000", 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4454e-05 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571058+0000", 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0145e-05 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656654+0000", 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085595579000000005 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656758+0000", 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010446999999999999 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656942+0000", 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183511 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.935 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26035 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6836224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 34.270749180000003, 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086157396999999997, 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:50.936 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26035 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571805+0000", 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000730326 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571812+0000", 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571833+0000", 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1506999999999999e-05 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571846+0000", 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2330999999999999e-05 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657009+0000", 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085162977000000001 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657091+0000", 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2514000000000003e-05 2024-09-21T23:16:50.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657232+0000", 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140858 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:50.940 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:50.942 INFO:teuthology.orchestra.run.smithi170.stdout:3549 write 0xba41800 thru 0xba5152e (0xfd2f bytes) 2024-09-21T23:16:50.942 INFO:teuthology.orchestra.run.smithi170.stdout:3555 read 0x75d3e00 thru 0x75dde64 (0xa065 bytes) 2024-09-21T23:16:50.942 INFO:teuthology.orchestra.run.smithi170.stdout:3557 punch from 0x278fe00 to 0x279fc00, (0xfe00 bytes) 2024-09-21T23:16:50.942 INFO:teuthology.orchestra.run.smithi170.stdout:3560 trunc from 0xba51600 to 0xc5f6000 2024-09-21T23:16:50.942 INFO:teuthology.orchestra.run.smithi170.stdout:3561 write 0x25a9e00 thru 0x25b6479 (0xc67a bytes) 2024-09-21T23:16:50.943 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:50.943 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:50.940+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:50.961 INFO:teuthology.orchestra.run.smithi170.stdout:3562 write 0xd762e00 thru 0xd76eacf (0xbcd0 bytes) 2024-09-21T23:16:50.963 INFO:teuthology.orchestra.run.smithi170.stdout:3565 read 0x2ad4400 thru 0x2ad96da (0x52db bytes) 2024-09-21T23:16:50.963 INFO:teuthology.orchestra.run.smithi170.stdout:3566 punch from 0xa9a6000 to 0xa9aa400, (0x4400 bytes) 2024-09-21T23:16:50.964 INFO:teuthology.orchestra.run.smithi170.stdout:3567 read 0x45be600 thru 0x45c2ba7 (0x45a8 bytes) 2024-09-21T23:16:50.978 INFO:teuthology.orchestra.run.smithi170.stdout:3569 trunc from 0xd76ec00 to 0x63ace00 2024-09-21T23:16:51.021 INFO:teuthology.orchestra.run.smithi170.stdout:3570 read 0x2691200 thru 0x269be01 (0xac02 bytes) 2024-09-21T23:16:51.024 INFO:teuthology.orchestra.run.smithi170.stdout:3574 write 0x2c06200 thru 0x2c09cdf (0x3ae0 bytes) 2024-09-21T23:16:51.024 INFO:teuthology.orchestra.run.smithi170.stdout:3576 read 0x5f62e00 thru 0x5f70604 (0xd805 bytes) 2024-09-21T23:16:51.024 INFO:teuthology.orchestra.run.smithi170.stdout:3577 read 0x55cde00 thru 0x55d8fdf (0xb1e0 bytes) 2024-09-21T23:16:51.026 INFO:teuthology.orchestra.run.smithi170.stdout:3578 trunc from 0x63ace00 to 0x6272200 2024-09-21T23:16:51.042 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:51.040+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:51.054 INFO:teuthology.orchestra.run.smithi170.stdout:3580 read 0x2065800 thru 0x207552e (0xfd2f bytes) 2024-09-21T23:16:51.057 INFO:teuthology.orchestra.run.smithi170.stdout:3582 write 0x5ed8600 thru 0x5ee747f (0xee80 bytes) 2024-09-21T23:16:51.057 INFO:teuthology.orchestra.run.smithi170.stdout:3583 read 0xddf400 thru 0xde751d (0x811e bytes) 2024-09-21T23:16:51.058 INFO:teuthology.orchestra.run.smithi170.stdout:3584 read 0x1faf200 thru 0x1fb5440 (0x6241 bytes) 2024-09-21T23:16:51.058 INFO:teuthology.orchestra.run.smithi170.stdout:3585 read 0x2ca6000 thru 0x2ca8ce3 (0x2ce4 bytes) 2024-09-21T23:16:51.099 INFO:teuthology.orchestra.run.smithi170.stdout:3588 write 0x8d4ba00 thru 0x8d55f26 (0xa527 bytes) 2024-09-21T23:16:51.100 INFO:teuthology.orchestra.run.smithi170.stdout:3591 read 0x3478c00 thru 0x3485fe8 (0xd3e9 bytes) 2024-09-21T23:16:51.101 INFO:teuthology.orchestra.run.smithi170.stdout:3593 read 0x24b8200 thru 0x24bb748 (0x3549 bytes) 2024-09-21T23:16:51.102 INFO:teuthology.orchestra.run.smithi170.stdout:3596 read 0x1ee3a00 thru 0x1ef3358 (0xf959 bytes) 2024-09-21T23:16:51.103 INFO:teuthology.orchestra.run.smithi170.stdout:3597 read 0x2cf4400 thru 0x2d02193 (0xdd94 bytes) 2024-09-21T23:16:51.104 INFO:teuthology.orchestra.run.smithi170.stdout:3599 read 0x43c5400 thru 0x43d087b (0xb47c bytes) 2024-09-21T23:16:51.104 INFO:teuthology.orchestra.run.smithi170.stdout:3602 write 0x40fe200 thru 0x40ff6f2 (0x14f3 bytes) 2024-09-21T23:16:51.110 INFO:teuthology.orchestra.run.smithi170.stdout:3603 trunc from 0x8d56000 to 0x6e1aa00 2024-09-21T23:16:51.119 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:51.120 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:51.120 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:51.120 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:51.122 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:16:51.123 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.2. 2024-09-21T23:16:51.137 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:51.159 INFO:teuthology.orchestra.run.smithi170.stdout:3604 trunc from 0x6e1aa00 to 0x4335c00 2024-09-21T23:16:51.178 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:51.176+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:51.184 INFO:teuthology.orchestra.run.smithi170.stdout:3607 write 0x2800e00 thru 0x2809759 (0x895a bytes) 2024-09-21T23:16:51.186 INFO:teuthology.orchestra.run.smithi170.stdout:3610 read 0x3c2bc00 thru 0x3c33960 (0x7d61 bytes) 2024-09-21T23:16:51.199 INFO:teuthology.orchestra.run.smithi170.stdout:3613 trunc from 0x4335c00 to 0x8976600 2024-09-21T23:16:51.216 INFO:teuthology.orchestra.run.smithi170.stdout:3615 punch from 0x21d1a00 to 0x21d7000, (0x5600 bytes) 2024-09-21T23:16:51.218 INFO:teuthology.orchestra.run.smithi170.stdout:3617 read 0x37f5e00 thru 0x3801ffb (0xc1fc bytes) 2024-09-21T23:16:51.218 INFO:teuthology.orchestra.run.smithi170.stdout:3618 read 0x2c91200 thru 0x2c9a4dc (0x92dd bytes) 2024-09-21T23:16:51.219 INFO:teuthology.orchestra.run.smithi170.stdout:3621 read 0x29b4400 thru 0x29bbe6d (0x7a6e bytes) 2024-09-21T23:16:51.219 INFO:teuthology.orchestra.run.smithi170.stdout:3622 punch from 0x6e6e600 to 0x6e6f000, (0xa00 bytes) 2024-09-21T23:16:51.227 INFO:teuthology.orchestra.run.smithi170.stdout:3623 trunc from 0x8976600 to 0x44f3e00 2024-09-21T23:16:51.244 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:51.240+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:51.266 INFO:teuthology.orchestra.run.smithi170.stdout:3625 trunc from 0x44f3e00 to 0x8c68c00 2024-09-21T23:16:51.291 INFO:teuthology.orchestra.run.smithi170.stdout:3626 read 0x47c3800 thru 0x47cb7d4 (0x7fd5 bytes) 2024-09-21T23:16:51.307 INFO:teuthology.orchestra.run.smithi170.stdout:3627 trunc from 0x8c68c00 to 0x176fe00 2024-09-21T23:16:51.310 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:51.310 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:51.310 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:51.310 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:51.310 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:51.327 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:51.345 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:51.343+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:51.359 INFO:teuthology.orchestra.run.smithi170.stdout:3628 write 0x2612a00 thru 0x261d3ac (0xa9ad bytes) 2024-09-21T23:16:51.407 INFO:teuthology.orchestra.run.smithi170.stdout:3630 write 0xd678000 thru 0xd67d926 (0x5927 bytes) 2024-09-21T23:16:51.408 INFO:teuthology.orchestra.run.smithi170.stdout:3631 punch from 0xa19c200 to 0xa19d200, (0x1000 bytes) 2024-09-21T23:16:51.409 INFO:teuthology.orchestra.run.smithi170.stdout:3633 read 0x97c9800 thru 0x97d90dc (0xf8dd bytes) 2024-09-21T23:16:51.409 INFO:teuthology.orchestra.run.smithi170.stdout:3635 clone 24 order 25 su 1048576 sc 11 2024-09-21T23:16:51.446 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:51.443+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:51.497 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:51.497 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.373903083999998, 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:16:51.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:51.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.503 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10124 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6156288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369545445999997, 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038330863999999999, 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.504 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10124 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0399999999999995e-07 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125818+0000", 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125827+0000", 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6330000000000002e-06 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125834+0000", 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9199999999999998e-06 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.163896+0000", 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038062072000000002 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164009+0000", 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011386400000000001 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164145+0000", 2024-09-21T23:16:51.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013601100000000001 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10125 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6141952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369507075999998, 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038589275999999999, 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.509 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10125 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:51.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125854+0000", 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125856+0000", 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8159999999999998e-06 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125865+0000", 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7739999999999993e-06 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125872+0000", 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164189+0000", 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038317276999999997 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164304+0000", 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114676 2024-09-21T23:16:51.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164442+0000", 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013814 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10126 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6172672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.514 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369479292999998, 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038865093000000003, 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10126 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:51.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125884+0000", 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125893+0000", 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-21T23:16:51.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125900+0000", 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0500000000000003e-06 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164491+0000", 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038591473000000001 2024-09-21T23:16:51.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164606+0000", 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115109 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164746+0000", 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013963800000000001 2024-09-21T23:16:51.519 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369451927, 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:51.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:16:51.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.524 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369424453999997, 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:16:51.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:51.530 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369384281000002, 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:16:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:16:51.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.535 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369333511999997, 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:16:51.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369302929, 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:51.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:16:51.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369275023, 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:16:51.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369247803, 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.551 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:51.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:16:51.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369220884000001, 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:16:51.556 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:51.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:16:51.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369165121000002, 2024-09-21T23:16:51.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:16:51.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.566 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369131948000003, 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.567 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:16:51.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369086729000003, 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369047449, 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:16:51.577 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:51.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:16:51.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.369016963, 2024-09-21T23:16:51.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:51.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:16:51.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.368987670000003, 2024-09-21T23:16:51.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:16:51.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.592 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.368945328000002, 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:16:51.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.368562132999998, 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:51.598 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:51.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:51.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:16:51.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.603 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:51.603 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:51.603 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:51.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:51.605 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:51.603+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:51.705 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:51.703+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:51.741 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:51.741 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:51.758 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:51.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:51.805 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:51.803+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:51.906 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:51.903+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:51.932 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:51.932 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:51.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:51.953 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:51.953 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:51.970 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:51.970 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:51.988 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:51.989 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:52.007 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:52.003+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:52.010 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:52.010 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:52.031 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:52.054 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:52.054 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:52.071 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:52.071 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:52.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:52.127 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:52.127+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:52.209 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:52.204+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:52.253 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:52.253 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:52.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:52.311 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:52.311+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:52.397 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:52.397 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:52.410 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:52.407+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:52.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:52.511 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:52.507+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:52.539 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-21T23:16:52.582 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:52.583 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:52.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:52.611 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:52.608+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:52.712 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:52.711+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:52.768 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:52.768 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:52.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:52.823 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:52.823+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:52.914 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:52.912+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:52.947 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:52.947 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:52.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:53.014 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:53.012+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:53.105 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:53.105 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:53.115 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:53.111+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:53.123 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:53.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:16:53.215 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:53.212+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:53.266 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:53.266 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:53.281 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:53.316 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:53.315+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:53.417 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:53.412+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:53.452 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:53.452 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:53.467 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:53.519 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:53.515+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:53.619 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:53.619+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:53.645 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:53.646 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:53.661 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:53.720 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:53.719+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:53.765 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:16:53.796 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:53.796 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:53.811 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:53.858 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:53.856+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:53.923 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:53.923+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:53.933 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:53.934 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:53.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:54.034 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:54.032+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:54.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:54.114 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:54.114 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:54.124 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:54.123+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:54.128 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:54.224 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:54.223+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:54.231 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:54.232 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:54.248 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:54.248 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:54.264 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:54.265 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:54.285 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:54.285 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:54.296 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:54.296 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:54.306 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:54.307 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:54.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:54.325 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:54.323+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:54.327 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:54.327 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:54.426 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:54.423+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:54.433 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:54.433 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:54.433 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:54.447 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:54.527 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:54.523+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:54.574 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:54.575 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:16:54.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:16:54.575 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:16:54.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.575 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:16:54.575 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:54.575 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 46.449096312999998, 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:16:54.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:54.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:16:54.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:16:54.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:16:54.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:16:54.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:16:54.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.584 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10124 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6156288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444738675000004, 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038330863999999999, 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10124 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0399999999999995e-07 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125818+0000", 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125827+0000", 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6330000000000002e-06 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125834+0000", 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9199999999999998e-06 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.163896+0000", 2024-09-21T23:16:54.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038062072000000002 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164009+0000", 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011386400000000001 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164145+0000", 2024-09-21T23:16:54.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013601100000000001 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10125 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6141952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444700304999998, 2024-09-21T23:16:54.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038589275999999999, 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10125 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125854+0000", 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125856+0000", 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8159999999999998e-06 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125865+0000", 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7739999999999993e-06 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125872+0000", 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164189+0000", 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038317276999999997 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164304+0000", 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114676 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164442+0000", 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013814 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10126 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6172672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:54.597 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444672521999998, 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038865093000000003, 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10126 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125884+0000", 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125893+0000", 2024-09-21T23:16:54.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125900+0000", 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0500000000000003e-06 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164491+0000", 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038591473000000001 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164606+0000", 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115109 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164746+0000", 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013963800000000001 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:54.603 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444645156, 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.604 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.609 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444617682999997, 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.610 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:16:54.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:16:54.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:16:54.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444577510000002, 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.617 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.622 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444526740999997, 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.623 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:16:54.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.628 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444496158, 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.629 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:16:54.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:16:54.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:16:54.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444468252, 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.634 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:16:54.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:16:54.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:16:54.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:16:54.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444441032, 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.640 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:54.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:16:54.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:16:54.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.646 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444414113000001, 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:16:54.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:16:54.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:16:54.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.652 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444358350000002, 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:16:54.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:16:54.657 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444325177000003, 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.658 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:16:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444279958000003, 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:16:54.664 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:16:54.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:16:54.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444240678, 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:16:54.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444210192, 2024-09-21T23:16:54.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:16:54.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:16:54.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:16:54.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444180899000003, 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.681 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:54.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.444138557000002, 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:16:54.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.443755361999997, 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:16:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:16:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:16:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:16:54.696 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:54.698 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:54.698 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:54.696+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:54.799 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:54.795+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5312.0:3579 3.1b 3.73e940bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:54.846310+0000", 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.8314999999999996e-05, 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000164761, 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:54.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5312", 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/687629543", 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3579 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846310+0000", 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846310+0000", 2024-09-21T23:16:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846311+0000", 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5370000000000001e-06 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846319+0000", 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1529999999999997e-06 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846343+0000", 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4199e-05 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846355+0000", 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1430000000000001e-05 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846412+0000", 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7465e-05 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846477+0000", 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4448000000000004e-05 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5312.0:3581 3.1b 3:dd0297ce:::rbd_data.10f1c0888e36.0000000000000307:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:54.846438+0000", 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.99997, 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032496800000000002, 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5312", 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/687629543", 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3581 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846438+0000", 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846438+0000", 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846439+0000", 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.37e-06 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846446+0000", 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9700000000000002e-06 2024-09-21T23:16:54.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846462+0000", 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6082999999999998e-05 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846472+0000", 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4960000000000002e-06 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846576+0000", 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104419 2024-09-21T23:16:54.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:54.846790+0000", 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000214157 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 2 2024-09-21T23:16:54.852 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:54.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:54.900 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:54.896+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:55.001 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:54.996+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:55.045 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:55.046 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:55.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:55.101 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:55.100+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:55.191 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:55.191 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:55.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.029809767000003, 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:16:55.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:16:55.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:16:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:16:55.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 48.964555009000001, 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.567212877000003, 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:16:55.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:16:55.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:16:55.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.563845188000002, 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:16:55.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:16:55.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.559735607999997, 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.555942686999998, 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:16:55.217 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:16:55.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.495783879999998, 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:16:55.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:16:55.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:16:55.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:55.226 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.492026809000002, 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:16:55.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:16:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.492000007999998, 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:16:55.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491973842999997, 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:55.239 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491945393000002, 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:16:55.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491854685, 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:16:55.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:16:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491823105999998, 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:16:55.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491785415999999, 2024-09-21T23:16:55.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:16:55.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.256 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491755026, 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:55.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491711608999999, 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:16:55.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:16:55.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:16:55.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.491676984000001, 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:16:55.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.269 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.488423484000002, 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:16:55.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.488388901999997, 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:16:55.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:16:55.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.272644503000002, 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:16:55.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:16:55.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.286 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:55.287 INFO:teuthology.orchestra.run.smithi170.stdout:3636 read 0x95dbe00 thru 0x95e8a94 (0xcc95 bytes) 2024-09-21T23:16:55.287 INFO:teuthology.orchestra.run.smithi170.stdout:3638 punch from 0xc4f0400 to 0xc4f3e00, (0x3a00 bytes) 2024-09-21T23:16:55.287 INFO:teuthology.orchestra.run.smithi170.stdout:3639 trunc from 0xd67da00 to 0x84d3000 2024-09-21T23:16:55.288 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:55.287+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:55.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:55.350 INFO:teuthology.orchestra.run.smithi170.stdout:3640 trunc from 0x84d3000 to 0xcd64000 2024-09-21T23:16:55.373 INFO:teuthology.orchestra.run.smithi170.stdout:3641 read 0x447da00 thru 0x44852d8 (0x78d9 bytes) 2024-09-21T23:16:55.376 INFO:teuthology.orchestra.run.smithi170.stdout:3642 write 0x1e65400 thru 0x1e72ec7 (0xdac8 bytes) 2024-09-21T23:16:55.377 INFO:teuthology.orchestra.run.smithi170.stdout:3644 punch from 0x8632c00 to 0x8637a00, (0x4e00 bytes) 2024-09-21T23:16:55.378 INFO:teuthology.orchestra.run.smithi170.stdout:3650 write 0xb7a5000 thru 0xb7aeb1b (0x9b1c bytes) 2024-09-21T23:16:55.379 INFO:teuthology.orchestra.run.smithi170.stdout:3651 write 0x4a03000 thru 0x4a0371c (0x71d bytes) 2024-09-21T23:16:55.387 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:55.387+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:55.408 INFO:teuthology.orchestra.run.smithi170.stdout:3652 trunc from 0xcd64000 to 0x5121a00 2024-09-21T23:16:55.480 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:55.480 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:55.480 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:55.480 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:55.488 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:55.487+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:55.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:55.496 INFO:teuthology.orchestra.run.smithi170.stdout:3656 trunc from 0x5121a00 to 0x296200 2024-09-21T23:16:55.589 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:55.584+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:55.609 INFO:teuthology.orchestra.run.smithi170.stdout:3657 read 0x4de00 thru 0x59af2 (0xbcf3 bytes) 2024-09-21T23:16:55.615 INFO:teuthology.orchestra.run.smithi170.stdout:3661 read 0x33600 thru 0x37bac (0x45ad bytes) 2024-09-21T23:16:55.652 INFO:teuthology.orchestra.run.smithi170.stdout:3662 write 0x4fe4800 thru 0x4fe90c5 (0x48c6 bytes) 2024-09-21T23:16:55.654 INFO:teuthology.orchestra.run.smithi170.stdout:3664 punch from 0x75200 to 0x78600, (0x3400 bytes) 2024-09-21T23:16:55.677 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:55.677 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:55.690 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:55.687+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:55.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:55.693 INFO:teuthology.orchestra.run.smithi170.stdout:3665 write 0xa8cf000 thru 0xa8d9082 (0xa083 bytes) 2024-09-21T23:16:55.694 INFO:teuthology.orchestra.run.smithi170.stdout:3666 punch from 0x2a0da00 to 0x2a1a600, (0xcc00 bytes) 2024-09-21T23:16:55.695 INFO:teuthology.orchestra.run.smithi170.stdout:3667 read 0xa00c200 thru 0xa00c25f (0x60 bytes) 2024-09-21T23:16:55.695 INFO:teuthology.orchestra.run.smithi170.stdout:3673 punch from 0x3274000 to 0x3276400, (0x2400 bytes) 2024-09-21T23:16:55.699 INFO:teuthology.orchestra.run.smithi170.stdout:3674 trunc from 0xa8d9200 to 0xc49da00 2024-09-21T23:16:55.729 INFO:teuthology.orchestra.run.smithi170.stdout:3675 punch from 0x82b7000 to 0x82c1e00, (0xae00 bytes) 2024-09-21T23:16:55.731 INFO:teuthology.orchestra.run.smithi170.stdout:3679 punch from 0x6f98800 to 0x6fa1600, (0x8e00 bytes) 2024-09-21T23:16:55.731 INFO:teuthology.orchestra.run.smithi170.stdout:3680 punch from 0x5111e00 to 0x511ac00, (0x8e00 bytes) 2024-09-21T23:16:55.731 INFO:teuthology.orchestra.run.smithi170.stdout:3682 punch from 0x935d200 to 0x935f000, (0x1e00 bytes) 2024-09-21T23:16:55.731 INFO:teuthology.orchestra.run.smithi170.stdout:3683 punch from 0x3451000 to 0x3457c00, (0x6c00 bytes) 2024-09-21T23:16:55.731 INFO:teuthology.orchestra.run.smithi170.stdout:3684 read 0x762b600 thru 0x763192b (0x632c bytes) 2024-09-21T23:16:55.731 INFO:teuthology.orchestra.run.smithi170.stdout:3685 read 0xd0e400 thru 0xd0ffd2 (0x1bd3 bytes) 2024-09-21T23:16:55.732 INFO:teuthology.orchestra.run.smithi170.stdout:3686 read 0x366b400 thru 0x3675ebb (0xaabc bytes) 2024-09-21T23:16:55.732 INFO:teuthology.orchestra.run.smithi170.stdout:3687 write 0x87aae00 thru 0x87ab96a (0xb6b bytes) 2024-09-21T23:16:55.739 INFO:teuthology.orchestra.run.smithi170.stdout:3689 trunc from 0xc49da00 to 0x8e0b000 2024-09-21T23:16:55.770 INFO:teuthology.orchestra.run.smithi170.stdout:3690 write 0x4ed5800 thru 0x4ee09af (0xb1b0 bytes) 2024-09-21T23:16:55.772 INFO:teuthology.orchestra.run.smithi170.stdout:3691 write 0x10d6600 thru 0x10da253 (0x3c54 bytes) 2024-09-21T23:16:55.790 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:55.788+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:55.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:16:55.807 INFO:teuthology.orchestra.run.smithi170.stdout:3694 write 0xbe90c00 thru 0xbe9ab38 (0x9f39 bytes) 2024-09-21T23:16:55.810 INFO:teuthology.orchestra.run.smithi170.stdout:3697 write 0x4db8200 thru 0x4db860a (0x40b bytes) 2024-09-21T23:16:55.812 INFO:teuthology.orchestra.run.smithi170.stdout:3698 write 0xb3bf400 thru 0xb3ca2d7 (0xaed8 bytes) 2024-09-21T23:16:55.813 INFO:teuthology.orchestra.run.smithi170.stdout:3699 read 0x31f4e00 thru 0x3203be4 (0xede5 bytes) 2024-09-21T23:16:55.814 INFO:teuthology.orchestra.run.smithi170.stdout:3702 read 0x3195000 thru 0x31a1cde (0xccdf bytes) 2024-09-21T23:16:55.821 INFO:teuthology.orchestra.run.smithi170.stdout:3703 trunc from 0xbe9ac00 to 0x9b41000 2024-09-21T23:16:55.852 INFO:teuthology.orchestra.run.smithi170.stdout:3704 read 0x5cee000 thru 0x5cf5cbb (0x7cbc bytes) 2024-09-21T23:16:55.855 INFO:teuthology.orchestra.run.smithi170.stdout:3707 write 0x7e6ac00 thru 0x7e6b099 (0x49a bytes) 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108720302000002, 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:55.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:16:55.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:55.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108675044999998, 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:55.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108625422999999, 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108595727999997, 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:55.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108562800000001, 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:55.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108528503999999, 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:55.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:55.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.898 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108503998000003, 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:55.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.903 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108476007999997, 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:55.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:55.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108452980000003, 2024-09-21T23:16:55.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:16:55.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108431469999999, 2024-09-21T23:16:55.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108395928999997, 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.918 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108368292000002, 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:55.927 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108333127999998, 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:55.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:55.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:16:55.931 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108298097000002, 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.932 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:55.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:55.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108262711999998, 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.941 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108222529999999, 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:55.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.946 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108185401999997, 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108145327000003, 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.951 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:55.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:55.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108120782999997, 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:55.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:55.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:16:55.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.108082998, 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:55.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:55.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:55.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:55.965 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:55.965 INFO:teuthology.orchestra.run.smithi170.stdout:3708 trunc from 0x9b41000 to 0x14d3400 2024-09-21T23:16:55.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:55.967 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:55.964+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:55.976 INFO:teuthology.orchestra.run.smithi170.stdout:3709 write 0xcedf600 thru 0xcee4cba (0x56bb bytes) 2024-09-21T23:16:55.978 INFO:teuthology.orchestra.run.smithi170.stdout:skipping zero length punch hole 2024-09-21T23:16:55.987 INFO:teuthology.orchestra.run.smithi170.stdout:3711 trunc from 0xcee4e00 to 0x9a1ee00 2024-09-21T23:16:56.017 INFO:teuthology.orchestra.run.smithi170.stdout:3713 punch from 0x999fc00 to 0x99a1600, (0x1a00 bytes) 2024-09-21T23:16:56.019 INFO:teuthology.orchestra.run.smithi170.stdout:3717 punch from 0x2599400 to 0x259bc00, (0x2800 bytes) 2024-09-21T23:16:56.022 INFO:teuthology.orchestra.run.smithi170.stdout:3718 trunc from 0x9a1ee00 to 0xb08c600 2024-09-21T23:16:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:3724 read 0x369c600 thru 0x36a9e9b (0xd89c bytes) 2024-09-21T23:16:56.063 INFO:teuthology.orchestra.run.smithi170.stdout:3725 trunc from 0xb08c600 to 0x449ca00 2024-09-21T23:16:56.066 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:56.063+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:56.100 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:56.101 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:56.103 INFO:teuthology.orchestra.run.smithi170.stdout:3726 read 0x2945e00 thru 0x29480db (0x22dc bytes) 2024-09-21T23:16:56.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:56.137 INFO:teuthology.orchestra.run.smithi170.stdout:3729 write 0x74f2800 thru 0x74f54b2 (0x2cb3 bytes) 2024-09-21T23:16:56.139 INFO:teuthology.orchestra.run.smithi170.stdout:3733 read 0x1096200 thru 0x1098045 (0x1e46 bytes) 2024-09-21T23:16:56.140 INFO:teuthology.orchestra.run.smithi170.stdout:3735 read 0x6bb5c00 thru 0x6bbc4be (0x68bf bytes) 2024-09-21T23:16:56.141 INFO:teuthology.orchestra.run.smithi170.stdout:3737 write 0x51dc600 thru 0x51dedeb (0x27ec bytes) 2024-09-21T23:16:56.142 INFO:teuthology.orchestra.run.smithi170.stdout:3738 read 0x6859200 thru 0x6867d17 (0xeb18 bytes) 2024-09-21T23:16:56.145 INFO:teuthology.orchestra.run.smithi170.stdout:3739 trunc from 0x74f5600 to 0x7fa3a00 2024-09-21T23:16:56.154 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:16:56.154 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:16:56.155 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T23:16:56.155 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:16:56.155 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:16:56.155 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-21T23:16:56.171 INFO:teuthology.orchestra.run.smithi170.stdout:3741 read 0x3b11c00 thru 0x3b15b46 (0x3f47 bytes) 2024-09-21T23:16:56.173 INFO:teuthology.orchestra.run.smithi170.stdout:3742 write 0x233c000 thru 0x2340af5 (0x4af6 bytes) 2024-09-21T23:16:56.194 INFO:teuthology.orchestra.run.smithi170.stdout:3745 write 0x8eb9200 thru 0x8ebf899 (0x669a bytes) 2024-09-21T23:16:56.195 INFO:teuthology.orchestra.run.smithi170.stdout:3746 write 0x4e88c00 thru 0x4e8fc30 (0x7031 bytes) 2024-09-21T23:16:56.200 INFO:teuthology.orchestra.run.smithi170.stdout:3748 trunc from 0x8ebfa00 to 0x6cec600 2024-09-21T23:16:56.208 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:56.207+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:56.220 INFO:teuthology.orchestra.run.smithi170.stdout:3752 punch from 0x1a5ca00 to 0x1a5fc00, (0x3200 bytes) 2024-09-21T23:16:56.221 INFO:teuthology.orchestra.run.smithi170.stdout:3754 read 0x9d8e00 thru 0x9e7401 (0xe602 bytes) 2024-09-21T23:16:56.229 INFO:teuthology.orchestra.run.smithi170.stdout:3760 trunc from 0x6cec600 to 0xac49400 2024-09-21T23:16:56.242 INFO:teuthology.orchestra.run.smithi170.stdout:3761 read 0x6f5a200 thru 0x6f640d0 (0x9ed1 bytes) 2024-09-21T23:16:56.244 INFO:teuthology.orchestra.run.smithi170.stdout:3764 read 0x389c400 thru 0x389d3c2 (0xfc3 bytes) 2024-09-21T23:16:56.248 INFO:teuthology.orchestra.run.smithi170.stdout:3766 trunc from 0xac49400 to 0x8861200 2024-09-21T23:16:56.268 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:56.267+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:56.269 INFO:teuthology.orchestra.run.smithi170.stdout:3767 read 0x6aff400 thru 0x6b099c4 (0xa5c5 bytes) 2024-09-21T23:16:56.285 INFO:teuthology.orchestra.run.smithi170.stdout:3768 trunc from 0x8861200 to 0x2d9ac00 2024-09-21T23:16:56.296 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:56.296 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:16:56.296 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:56.300 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:56.300 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:56.300 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:56.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:56.332 INFO:teuthology.orchestra.run.smithi170.stdout:3769 write 0x5417a00 thru 0x5422f46 (0xb547 bytes) 2024-09-21T23:16:56.333 INFO:teuthology.orchestra.run.smithi170.stdout:3771 read 0x3712400 thru 0x37191bd (0x6dbe bytes) 2024-09-21T23:16:56.337 INFO:teuthology.orchestra.run.smithi170.stdout:3772 read 0x73a00 thru 0x783e9 (0x49ea bytes) 2024-09-21T23:16:56.341 INFO:teuthology.orchestra.run.smithi170.stdout:3773 punch from 0x4b96800 to 0x4ba4e00, (0xe600 bytes) 2024-09-21T23:16:56.341 INFO:teuthology.orchestra.run.smithi170.stdout:3774 read 0x702a00 thru 0x7035e7 (0xbe8 bytes) 2024-09-21T23:16:56.342 INFO:teuthology.orchestra.run.smithi170.stdout:3775 trunc from 0x5423000 to 0x4ef4c00 2024-09-21T23:16:56.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:16:56.369 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:56.364+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:56.386 INFO:teuthology.orchestra.run.smithi170.stdout:3776 write 0x56c4800 thru 0x56cd665 (0x8e66 bytes) 2024-09-21T23:16:56.425 INFO:teuthology.orchestra.run.smithi170.stdout:3780 write 0xa73c600 thru 0xa74676f (0xa170 bytes) 2024-09-21T23:16:56.428 INFO:teuthology.orchestra.run.smithi170.stdout:3781 read 0x39ed200 thru 0x39fb1e1 (0xdfe2 bytes) 2024-09-21T23:16:56.429 INFO:teuthology.orchestra.run.smithi170.stdout:3784 write 0x65bd000 thru 0x65c7c53 (0xac54 bytes) 2024-09-21T23:16:56.459 INFO:teuthology.orchestra.run.smithi170.stdout:3789 write 0xae09c00 thru 0xae1266a (0x8a6b bytes) 2024-09-21T23:16:56.470 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:56.468+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.563271258, 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:56.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:56.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.562250410999994, 2024-09-21T23:16:56.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:56.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:56.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.562190737999998, 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:56.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.505 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.562100762, 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:56.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.562050851999999, 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.511 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:56.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561994712000001, 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:56.516 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:56.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:56.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.521 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561940929000002, 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:56.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:56.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561908653000003, 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:56.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:56.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:56.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561878372999999, 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.536 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561848717000004, 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:56.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:56.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:56.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.56181875, 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:56.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561740920999995, 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:56.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561693214000002, 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:56.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561663018000004, 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:56.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:56.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561633100999998, 2024-09-21T23:16:56.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:56.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561603301000005, 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561573367999998, 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561508404999998, 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561477929000006, 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.561448764999994, 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:56.595 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:56.596 INFO:teuthology.orchestra.run.smithi170.stdout:3790 trunc from 0xae12800 to 0xddfe00 2024-09-21T23:16:56.596 INFO:teuthology.orchestra.run.smithi170.stdout:3791 write 0x9f2a000 thru 0x9f39262 (0xf263 bytes) 2024-09-21T23:16:56.596 INFO:teuthology.orchestra.run.smithi170.stdout:3792 read 0x9728c00 thru 0x972bce8 (0x30e9 bytes) 2024-09-21T23:16:56.596 INFO:teuthology.orchestra.run.smithi170.stdout:3793 read 0x523e200 thru 0x523f034 (0xe35 bytes) 2024-09-21T23:16:56.597 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:56.598 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:56.598 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:56.598 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:56.598 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:56.599 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:56.599 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:56.599 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:56.599 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:56.600 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:56.600 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:56.601 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:56.592+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:56.601 INFO:teuthology.orchestra.run.smithi170.stdout:3795 trunc from 0x9f39400 to 0x7ebb600 2024-09-21T23:16:56.607 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:56.607 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:56.627 INFO:teuthology.orchestra.run.smithi170.stdout:3796 read 0x68b7c00 thru 0x68ba9ed (0x2dee bytes) 2024-09-21T23:16:56.629 INFO:teuthology.orchestra.run.smithi170.stdout:3798 punch from 0x435c200 to 0x435ee00, (0x2c00 bytes) 2024-09-21T23:16:56.630 INFO:teuthology.orchestra.run.smithi170.stdout:3799 punch from 0x101ac00 to 0x101e800, (0x3c00 bytes) 2024-09-21T23:16:56.630 INFO:teuthology.orchestra.run.smithi170.stdout:3800 read 0x3f6aa00 thru 0x3f6d9cb (0x2fcc bytes) 2024-09-21T23:16:56.630 INFO:teuthology.orchestra.run.smithi170.stdout:3801 read 0x84b800 thru 0x859d99 (0xe59a bytes) 2024-09-21T23:16:56.632 INFO:teuthology.orchestra.run.smithi170.stdout:3802 write 0x6bb7000 thru 0x6bbc18b (0x518c bytes) 2024-09-21T23:16:56.632 INFO:teuthology.orchestra.run.smithi170.stdout:3805 write 0x416b400 thru 0x41704a2 (0x50a3 bytes) 2024-09-21T23:16:56.633 INFO:teuthology.orchestra.run.smithi170.stdout:3806 write 0x18f7600 thru 0x190257f (0xaf80 bytes) 2024-09-21T23:16:56.634 INFO:teuthology.orchestra.run.smithi170.stdout:3807 read 0x4c77a00 thru 0x4c825f5 (0xabf6 bytes) 2024-09-21T23:16:56.635 INFO:teuthology.orchestra.run.smithi170.stdout:3810 read 0x6ca0800 thru 0x6caae04 (0xa605 bytes) 2024-09-21T23:16:56.635 INFO:teuthology.orchestra.run.smithi170.stdout:3811 write 0x39f9000 thru 0x39fde67 (0x4e68 bytes) 2024-09-21T23:16:56.636 INFO:teuthology.orchestra.run.smithi170.stdout:3813 read 0x3d18600 thru 0x3d22c2d (0xa62e bytes) 2024-09-21T23:16:56.636 INFO:teuthology.orchestra.run.smithi170.stdout:3815 read 0x9e9400 thru 0x9f07ff (0x7400 bytes) 2024-09-21T23:16:56.680 INFO:teuthology.orchestra.run.smithi170.stdout:3816 write 0xeccda00 thru 0xecd9a23 (0xc024 bytes) 2024-09-21T23:16:56.682 INFO:teuthology.orchestra.run.smithi170.stdout:3817 read 0xb8b9e00 thru 0xb8be73e (0x493f bytes) 2024-09-21T23:16:56.682 INFO:teuthology.orchestra.run.smithi170.stdout:3818 write 0x6f28800 thru 0x6f2b662 (0x2e63 bytes) 2024-09-21T23:16:56.683 INFO:teuthology.orchestra.run.smithi170.stdout:3819 read 0x317e00 thru 0x31b418 (0x3619 bytes) 2024-09-21T23:16:56.684 INFO:teuthology.orchestra.run.smithi170.stdout:3820 read 0xcb91000 thru 0xcb9a920 (0x9921 bytes) 2024-09-21T23:16:56.685 INFO:teuthology.orchestra.run.smithi170.stdout:3822 write 0xb934000 thru 0xb9422af (0xe2b0 bytes) 2024-09-21T23:16:56.686 INFO:teuthology.orchestra.run.smithi170.stdout:3824 write 0x5812a00 thru 0x5819f53 (0x7554 bytes) 2024-09-21T23:16:56.687 INFO:teuthology.orchestra.run.smithi170.stdout:3826 write 0x28f8200 thru 0x28f848b (0x28c bytes) 2024-09-21T23:16:56.697 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:56.695+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:56.706 INFO:teuthology.orchestra.run.smithi170.stdout:3827 trunc from 0xecd9c00 to 0x773f400 2024-09-21T23:16:56.736 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:56.736 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:56.752 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:56.760 INFO:teuthology.orchestra.run.smithi170.stdout:3828 trunc from 0x773f400 to 0x1f7c200 2024-09-21T23:16:56.789 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:16:56.797 INFO:teuthology.orchestra.run.smithi170.stdout:3829 read 0x58ce00 thru 0x594d2b (0x7f2c bytes) 2024-09-21T23:16:56.797 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:56.795+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:56.847 INFO:teuthology.orchestra.run.smithi170.stdout:3833 write 0xe1e9000 thru 0xe1eadb6 (0x1db7 bytes) 2024-09-21T23:16:56.857 INFO:teuthology.orchestra.run.smithi170.stdout:3836 trunc from 0xe1eae00 to 0xb012200 2024-09-21T23:16:56.879 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:56.879 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:56.897 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:16:56.898 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:56.895+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:56.914 INFO:teuthology.orchestra.run.smithi170.stdout:3839 trunc from 0xb012200 to 0x643400 2024-09-21T23:16:56.977 INFO:teuthology.orchestra.run.smithi170.stdout:3842 punch from 0xd3200 to 0xdca00, (0x9800 bytes) 2024-09-21T23:16:56.999 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:56.995+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:57.002 INFO:teuthology.orchestra.run.smithi170.stdout:3843 write 0x260a600 thru 0x261025b (0x5c5c bytes) 2024-09-21T23:16:57.022 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:57.022 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:57.024 INFO:teuthology.orchestra.run.smithi170.stdout:3844 trunc from 0x2610400 to 0x97f3a00 2024-09-21T23:16:57.038 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:16:57.042 INFO:teuthology.orchestra.run.smithi170.stdout:3846 read 0x4525000 thru 0x452ce6f (0x7e70 bytes) 2024-09-21T23:16:57.046 INFO:teuthology.orchestra.run.smithi170.stdout:3847 read 0x4315e00 thru 0x431dff6 (0x81f7 bytes) 2024-09-21T23:16:57.056 INFO:teuthology.orchestra.run.smithi170.stdout:3849 trunc from 0x97f3a00 to 0xd62e800 2024-09-21T23:16:57.071 INFO:teuthology.orchestra.run.smithi170.stdout:3850 punch from 0x4bb3600 to 0x4bc1800, (0xe200 bytes) 2024-09-21T23:16:57.073 INFO:teuthology.orchestra.run.smithi170.stdout:3852 read 0x8856c00 thru 0x88669a3 (0xfda4 bytes) 2024-09-21T23:16:57.100 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:57.096+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:57.103 INFO:teuthology.orchestra.run.smithi170.stdout:3853 trunc from 0xd62e800 to 0x23dc800 2024-09-21T23:16:57.147 INFO:teuthology.orchestra.run.smithi170.stdout:3854 read 0xaa6c00 thru 0xaae482 (0x7883 bytes) 2024-09-21T23:16:57.148 INFO:teuthology.orchestra.run.smithi170.stdout:3855 read 0x22b1800 thru 0x22bb070 (0x9871 bytes) 2024-09-21T23:16:57.161 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:57.162 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:57.177 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:16:57.182 INFO:teuthology.orchestra.run.smithi170.stdout:3856 write 0xa7df200 thru 0xa7e1354 (0x2155 bytes) 2024-09-21T23:16:57.189 INFO:teuthology.orchestra.run.smithi170.stdout:3858 trunc from 0xa7e1400 to 0x95bbe00 2024-09-21T23:16:57.201 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:57.199+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:57.247 INFO:teuthology.orchestra.run.smithi170.stdout:3859 write 0xdda9400 thru 0xddad3a7 (0x3fa8 bytes) 2024-09-21T23:16:57.257 INFO:teuthology.orchestra.run.smithi170.stdout:3861 trunc from 0xddad400 to 0x9cb9c00 2024-09-21T23:16:57.284 INFO:teuthology.orchestra.run.smithi170.stdout:3862 read 0x93b6600 thru 0x93ba00f (0x3a10 bytes) 2024-09-21T23:16:57.287 INFO:teuthology.orchestra.run.smithi170.stdout:3863 read 0x19e9c00 thru 0x19f1f92 (0x8393 bytes) 2024-09-21T23:16:57.288 INFO:teuthology.orchestra.run.smithi170.stdout:3864 write 0x2bb4e00 thru 0x2bc454c (0xf74d bytes) 2024-09-21T23:16:57.288 INFO:teuthology.orchestra.run.smithi170.stdout:3865 read 0x2ca2800 thru 0x2caca3f (0xa240 bytes) 2024-09-21T23:16:57.290 INFO:teuthology.orchestra.run.smithi170.stdout:3866 punch from 0x1291600 to 0x1294600, (0x3000 bytes) 2024-09-21T23:16:57.301 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:57.300+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:57.305 INFO:teuthology.orchestra.run.smithi170.stdout:3867 trunc from 0x9cb9c00 to 0x2113600 2024-09-21T23:16:57.347 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:57.347 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:57.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:16:57.368 INFO:teuthology.orchestra.run.smithi170.stdout:3870 write 0x774fa00 thru 0x775d73f (0xdd40 bytes) 2024-09-21T23:16:57.395 INFO:teuthology.orchestra.run.smithi170.stdout:3871 write 0x8f6ee00 thru 0x8f7e476 (0xf677 bytes) 2024-09-21T23:16:57.402 INFO:teuthology.orchestra.run.smithi170.stdout:3874 read 0x7a03e00 thru 0x7a080b0 (0x42b1 bytes) 2024-09-21T23:16:57.403 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:57.399+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:57.403 INFO:teuthology.orchestra.run.smithi170.stdout:3877 read 0x4cd3200 thru 0x4cda908 (0x7709 bytes) 2024-09-21T23:16:57.404 INFO:teuthology.orchestra.run.smithi170.stdout:3879 write 0x8961400 thru 0x89666fc (0x52fd bytes) 2024-09-21T23:16:57.417 INFO:teuthology.orchestra.run.smithi170.stdout:3881 trunc from 0x8f7e600 to 0xd689600 2024-09-21T23:16:57.439 INFO:teuthology.orchestra.run.smithi170.stdout:3888 write 0x719d200 thru 0x71a886c (0xb66d bytes) 2024-09-21T23:16:57.441 INFO:teuthology.orchestra.run.smithi170.stdout:3892 write 0x8d7a00 thru 0x8e01b3 (0x87b4 bytes) 2024-09-21T23:16:57.441 INFO:teuthology.orchestra.run.smithi170.stdout:3893 punch from 0x5cb5200 to 0x5cb9000, (0x3e00 bytes) 2024-09-21T23:16:57.441 INFO:teuthology.orchestra.run.smithi170.stdout:3895 read 0xd97e00 thru 0xda14f1 (0x96f2 bytes) 2024-09-21T23:16:57.443 INFO:teuthology.orchestra.run.smithi170.stdout:3901 write 0x9bbc600 thru 0x9bbe73e (0x213f bytes) 2024-09-21T23:16:57.444 INFO:teuthology.orchestra.run.smithi170.stdout:3902 read 0x3ca4e00 thru 0x3cb40bd (0xf2be bytes) 2024-09-21T23:16:57.446 INFO:teuthology.orchestra.run.smithi170.stdout:3904 write 0x7bf400 thru 0x7cba35 (0xc636 bytes) 2024-09-21T23:16:57.446 INFO:teuthology.orchestra.run.smithi170.stdout:3908 write 0x62f9400 thru 0x6304a35 (0xb636 bytes) 2024-09-21T23:16:57.450 INFO:teuthology.orchestra.run.smithi170.stdout:3909 read 0x22c0600 thru 0x22cb02a (0xaa2b bytes) 2024-09-21T23:16:57.470 INFO:teuthology.orchestra.run.smithi170.stdout:3912 trunc from 0xd689600 to 0x23e6a00 2024-09-21T23:16:57.503 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:57.503+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:57.525 INFO:teuthology.orchestra.run.smithi170.stdout:3913 read 0x1ce3e00 thru 0x1ce3f04 (0x105 bytes) 2024-09-21T23:16:57.535 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:57.535 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:57.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:16:57.557 INFO:teuthology.orchestra.run.smithi170.stdout:3915 write 0x4c28000 thru 0x4c36693 (0xe694 bytes) 2024-09-21T23:16:57.560 INFO:teuthology.orchestra.run.smithi170.stdout:3916 read 0x2ccbc00 thru 0x2ccf6fa (0x3afb bytes) 2024-09-21T23:16:57.563 INFO:teuthology.orchestra.run.smithi170.stdout:3918 read 0x3e7a00 thru 0x3f5ff3 (0xe5f4 bytes) 2024-09-21T23:16:57.565 INFO:teuthology.orchestra.run.smithi170.stdout:3920 read 0xde3400 thru 0xde3aab (0x6ac bytes) 2024-09-21T23:16:57.566 INFO:teuthology.orchestra.run.smithi170.stdout:3921 clone 25 order 25 su 4194304 sc 2 2024-09-21T23:16:57.604 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:57.603+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:57.679 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:57.679 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:57.693 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:16:57.705 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:57.703+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:57.806 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:57.804+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:57.818 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:57.818 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:57.835 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:16:57.907 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:57.903+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:57.933 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone24 from 0x4c36800 (overlap 0x296200) to 0x84600 2024-09-21T23:16:58.007 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:58.004+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:58.008 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:58.008 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:58.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:16:58.108 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:58.104+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:58.163 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:58.163 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:58.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:16:58.209 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:58.204+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:58.307 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:58.307 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:58.309 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:58.307+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:58.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:16:58.410 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:58.407+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:58.497 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:58.497 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:58.511 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:16:58.507+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:16:58.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:16:58.590 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-21T23:16:58.612 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:58.608+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:58.627 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:16:58.676 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:16:58.676 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:16:58.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:16:58.735 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:58.735+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:58.774 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:16:58.774 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:16:58.792 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:16:58.792 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:16:58.814 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:58.812+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:58.816 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:16:58.825 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:16:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:16:58.830 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:58.830 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:58.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:16:58.847 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:16:58.848 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:16:58.869 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:16:58.869 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:16:58.906 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:16:58.906 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:16:58.915 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:16:58.911+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:16:58.975 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215672294000001, 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:58.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:16:58.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:16:58.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215627036999997, 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215577414999998, 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:58.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:16:58.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:16:58.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215547720000004, 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:58.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:16:58.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:16:58.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215514792, 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:58.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:58.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.998 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215480495999998, 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:58.999 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215455990000002, 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:16:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215428000000003, 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:16:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:16:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215404972000002, 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:16:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:16:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:16:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215383461999998, 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:16:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215347921000003, 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:16:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:16:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215320284000001, 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:16:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:16:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215285119999997, 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:16:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215250089000001, 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:16:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215214703999997, 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:16:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215174521999998, 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:16:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:16:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215137394000003, 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:16:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215097319000002, 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:16:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:16:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:16:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215072775000003, 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:16:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.215034989999999, 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:16:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:16:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:16:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.093 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:59.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:16:59.095 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:59.091+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:59.195 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:59.192+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:59.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:59.270 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:16:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:59.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:16:59.297 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:59.296+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:59.397 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:59.392+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:59.416 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:59.416 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:16:59.416 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:16:59.416 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:16:59.416 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:59.431 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:16:59.498 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:16:59.495+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.631733355999998, 2024-09-21T23:16:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:16:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630712509000006, 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:16:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630652835999996, 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630562859999998, 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:16:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630512949999996, 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:16:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:16:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630456809999998, 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:16:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:16:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:16:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:16:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:59.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630403027, 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:59.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630370751000001, 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:16:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:16:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630340470999997, 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:16:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:16:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630310815000001, 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630280847999998, 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630203019000007, 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:16:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630155311999999, 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:16:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630125116000002, 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:16:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:16:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:16:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630095198999996, 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:16:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:16:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630065399000003, 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.630035465999995, 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:16:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:16:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:16:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.629970502999996, 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:16:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.629940027000004, 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 84.629910863000006, 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.655 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:16:59.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:16:59.659 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:16:59.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:16:59.661 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:59.656+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:59.761 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:16:59.760+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:16:59.797 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:16:59.797 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:59.808 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:16:59.813 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:16:59.898 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:16:59.896+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:16:59.963 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:16:59.960+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:16:59.977 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:16:59.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:16:59.977 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:16:59.977 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:16:59.977 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:16:59.993 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:00.006 INFO:teuthology.orchestra.run.smithi170.stdout:3922 write 0xe8dba00 thru 0xe8e44c6 (0x8ac7 bytes) 2024-09-21T23:17:00.007 INFO:teuthology.orchestra.run.smithi170.stdout:3923 read 0x40a1000 thru 0x40a7ca0 (0x6ca1 bytes) 2024-09-21T23:17:00.011 INFO:teuthology.orchestra.run.smithi170.stdout:3924 read 0xc913200 thru 0xc9204b6 (0xd2b7 bytes) 2024-09-21T23:17:00.020 INFO:teuthology.orchestra.run.smithi170.stdout:3925 trunc from 0xe8e4600 to 0x9636c00 2024-09-21T23:17:00.055 INFO:teuthology.orchestra.run.smithi170.stdout:3927 flatten 2024-09-21T23:17:00.082 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:00.080+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:00.091 INFO:teuthology.orchestra.run.smithi170.stdout:3928 read 0x7ef4200 thru 0x7f017a8 (0xd5a9 bytes) 2024-09-21T23:17:00.091 INFO:teuthology.orchestra.run.smithi170.stdout:3933 read 0xf18c00 thru 0xf27f1d (0xf31e bytes) 2024-09-21T23:17:00.093 INFO:teuthology.orchestra.run.smithi170.stdout:3934 write 0x1a5c000 thru 0x1a5d532 (0x1533 bytes) 2024-09-21T23:17:00.094 INFO:teuthology.orchestra.run.smithi170.stdout:3935 read 0x56da800 thru 0x56ea67c (0xfe7d bytes) 2024-09-21T23:17:00.094 INFO:teuthology.orchestra.run.smithi170.stdout:3936 punch from 0x7191e00 to 0x719a200, (0x8400 bytes) 2024-09-21T23:17:00.095 INFO:teuthology.orchestra.run.smithi170.stdout:3937 punch from 0x3c32e00 to 0x3c3c000, (0x9200 bytes) 2024-09-21T23:17:00.095 INFO:teuthology.orchestra.run.smithi170.stdout:3938 read 0x3533a00 thru 0x353eb4b (0xb14c bytes) 2024-09-21T23:17:00.099 INFO:teuthology.orchestra.run.smithi170.stdout:3939 write 0x73c1200 thru 0x73c8c24 (0x7a25 bytes) 2024-09-21T23:17:00.102 INFO:teuthology.orchestra.run.smithi170.stdout:3940 write 0x9224a00 thru 0x922891a (0x3f1b bytes) 2024-09-21T23:17:00.102 INFO:teuthology.orchestra.run.smithi170.stdout:3943 read 0x8c04000 thru 0x8c05093 (0x1094 bytes) 2024-09-21T23:17:00.103 INFO:teuthology.orchestra.run.smithi170.stdout:3949 write 0xf9200 thru 0xf95ad (0x3ae bytes) 2024-09-21T23:17:00.103 INFO:teuthology.orchestra.run.smithi170.stdout:3950 read 0x45e200 thru 0x45ebdb (0x9dc bytes) 2024-09-21T23:17:00.104 INFO:teuthology.orchestra.run.smithi170.stdout:3952 write 0x621a000 thru 0x621ddad (0x3dae bytes) 2024-09-21T23:17:00.132 INFO:teuthology.orchestra.run.smithi170.stdout:3954 write 0xbbd6e00 thru 0xbbe2d79 (0xbf7a bytes) 2024-09-21T23:17:00.135 INFO:teuthology.orchestra.run.smithi170.stdout:3956 write 0x117e00 thru 0x125981 (0xdb82 bytes) 2024-09-21T23:17:00.137 INFO:teuthology.orchestra.run.smithi170.stdout:3958 trunc from 0xbbe2e00 to 0xbb66c00 2024-09-21T23:17:00.155 INFO:teuthology.orchestra.run.smithi170.stdout:3959 read 0x6430000 thru 0x6435ee1 (0x5ee2 bytes) 2024-09-21T23:17:00.156 INFO:teuthology.orchestra.run.smithi170.stdout:3961 write 0x905c000 thru 0x905f714 (0x3715 bytes) 2024-09-21T23:17:00.157 INFO:teuthology.orchestra.run.smithi170.stdout:3962 write 0x174ac00 thru 0x1756efe (0xc2ff bytes) 2024-09-21T23:17:00.159 INFO:teuthology.orchestra.run.smithi170.stdout:3963 read 0x2f88200 thru 0x2f8ed54 (0x6b55 bytes) 2024-09-21T23:17:00.165 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:00.160+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:00.185 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.965030824, 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:00.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:00.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.452625541, 2024-09-21T23:17:00.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:00.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:00.197 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.45256063799999, 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:00.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.203 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.452414199, 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:00.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:00.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:00.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:00.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.452345084, 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.209 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.45173275900001, 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:00.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:00.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.451528354, 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.221 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:00.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:00.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:00.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.226 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.45148556800001, 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.06052728, 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.232 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:00.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:00.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.06044077000001, 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.238 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:00.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:00.243 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.060363881, 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:00.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:00.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:00.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:00.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.06015904099999, 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:00.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:00.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:00.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.06007075900001, 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:00.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:00.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:00.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.059939931, 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:00.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.059894278, 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:00.266 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.05985585100001, 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.272 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:00.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:00.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:00.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.05972638899999, 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:00.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:00.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:00.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:00.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.05970474700001, 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.283 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:00.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:00.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.05968470299999, 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:00.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:00.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 133.059600711, 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.300 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:00.302 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:00.302 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:00.296+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:00.304 INFO:teuthology.orchestra.run.smithi170.stdout:3965 write 0xe9c8a00 thru 0xe9d3eff (0xb500 bytes) 2024-09-21T23:17:00.304 INFO:teuthology.orchestra.run.smithi170.stdout:3966 write 0xc8f0400 thru 0xc8fe5a4 (0xe1a5 bytes) 2024-09-21T23:17:00.304 INFO:teuthology.orchestra.run.smithi170.stdout:3967 trunc from 0xe9d4000 to 0xc455a00 2024-09-21T23:17:00.304 INFO:teuthology.orchestra.run.smithi170.stdout:3972 write 0x8ca8200 thru 0x8ca8469 (0x26a bytes) 2024-09-21T23:17:00.304 INFO:teuthology.orchestra.run.smithi170.stdout:3976 write 0x8e17c00 thru 0x8e239a3 (0xbda4 bytes) 2024-09-21T23:17:00.304 INFO:teuthology.orchestra.run.smithi170.stdout:3981 read 0x2b40e00 thru 0x2b4e076 (0xd277 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3983 write 0x39a6e00 thru 0x39adcd3 (0x6ed4 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3986 write 0xb843c00 thru 0xb84e248 (0xa649 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3987 read 0xc0b6c00 thru 0xc0c1ccd (0xb0ce bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3989 write 0x79f7a00 thru 0x7a02ddc (0xb3dd bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3992 write 0xb79de00 thru 0xb7a8fb7 (0xb1b8 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3993 write 0x6c5b400 thru 0x6c66d66 (0xb967 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3995 read 0x6422800 thru 0x6432128 (0xf929 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3996 read 0x2a91000 thru 0x2a915cd (0x5ce bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:3997 write 0xd760000 thru 0xd761146 (0x1147 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:4000 read 0x49c9600 thru 0x49cd8d9 (0x42da bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:4001 write 0x7ed9800 thru 0x7ee5d78 (0xc579 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:4005 trunc from 0xd761200 to 0xdaeda00 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:4007 read 0x98c0400 thru 0x98c6c69 (0x686a bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:4008 write 0xb70e00 thru 0xb75667 (0x4868 bytes) 2024-09-21T23:17:00.305 INFO:teuthology.orchestra.run.smithi170.stdout:4011 read 0xbad8600 thru 0xbadc091 (0x3a92 bytes) 2024-09-21T23:17:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:4012 write 0x5953200 thru 0x59605f0 (0xd3f1 bytes) 2024-09-21T23:17:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:4013 read 0xa5eaa00 thru 0xa5f0243 (0x5844 bytes) 2024-09-21T23:17:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:4014 write 0xc613c00 thru 0xc61de69 (0xa26a bytes) 2024-09-21T23:17:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:4015 read 0xaf13e00 thru 0xaf18272 (0x4473 bytes) 2024-09-21T23:17:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:4016 read 0x6ccea00 thru 0x6cdb6d2 (0xccd3 bytes) 2024-09-21T23:17:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:4017 read 0x2ce2200 thru 0x2ceef5d (0xcd5e bytes) 2024-09-21T23:17:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:4020 write 0xa51fa00 thru 0xa526ee4 (0x74e5 bytes) 2024-09-21T23:17:00.307 INFO:teuthology.orchestra.run.smithi170.stdout:4021 punch from 0x38bae00 to 0x38c4a00, (0x9c00 bytes) 2024-09-21T23:17:00.307 INFO:teuthology.orchestra.run.smithi170.stdout:4023 read 0xd09dc00 thru 0xd0ac8ad (0xecae bytes) 2024-09-21T23:17:00.307 INFO:teuthology.orchestra.run.smithi170.stdout:4024 write 0x1ebf400 thru 0x1ec7c45 (0x8846 bytes) 2024-09-21T23:17:00.307 INFO:teuthology.orchestra.run.smithi170.stdout:4025 read 0xcb50600 thru 0xcb59e39 (0x983a bytes) 2024-09-21T23:17:00.307 INFO:teuthology.orchestra.run.smithi170.stdout:4026 read 0x5b04400 thru 0x5b08be1 (0x47e2 bytes) 2024-09-21T23:17:00.307 INFO:teuthology.orchestra.run.smithi170.stdout:4027 trunc from 0xdaeda00 to 0xc313e00 2024-09-21T23:17:00.333 INFO:teuthology.orchestra.run.smithi170.stdout:4030 punch from 0x96f7c00 to 0x96fdc00, (0x6000 bytes) 2024-09-21T23:17:00.335 INFO:teuthology.orchestra.run.smithi170.stdout:4033 write 0x5065a00 thru 0x5071253 (0xb854 bytes) 2024-09-21T23:17:00.336 INFO:teuthology.orchestra.run.smithi170.stdout:4035 read 0x7760000 thru 0x77641a2 (0x41a3 bytes) 2024-09-21T23:17:00.354 INFO:teuthology.orchestra.run.smithi170.stdout:4046 trunc from 0xc313e00 to 0x60b9a00 2024-09-21T23:17:00.402 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:00.399+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:00.451 INFO:teuthology.orchestra.run.smithi170.stdout:4050 trunc from 0x60b9a00 to 0x71fe00 2024-09-21T23:17:00.474 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:00.474 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:00.474 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:00.474 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:00.487 INFO:teuthology.orchestra.run.smithi170.stdout:4051 read 0x1fea00 thru 0x205701 (0x6d02 bytes) 2024-09-21T23:17:00.490 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:00.491 INFO:teuthology.orchestra.run.smithi170.stdout:4052 read 0x269400 thru 0x26d406 (0x4007 bytes) 2024-09-21T23:17:00.521 INFO:teuthology.orchestra.run.smithi170.stdout:4053 trunc from 0x71fe00 to 0xc6b9400 2024-09-21T23:17:00.536 INFO:teuthology.orchestra.run.smithi170.stdout:4054 read 0x138e600 thru 0x139a988 (0xc389 bytes) 2024-09-21T23:17:00.537 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:00.536+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:00.538 INFO:teuthology.orchestra.run.smithi170.stdout:4055 read 0xa286800 thru 0xa28be26 (0x5627 bytes) 2024-09-21T23:17:00.538 INFO:teuthology.orchestra.run.smithi170.stdout:4057 punch from 0x2b7da00 to 0x2b8c800, (0xee00 bytes) 2024-09-21T23:17:00.562 INFO:teuthology.orchestra.run.smithi170.stdout:4059 trunc from 0xc6b9400 to 0x25b5400 2024-09-21T23:17:00.581 INFO:teuthology.orchestra.run.smithi170.stdout:4061 read 0x29ba00 thru 0x2a77a1 (0xbda2 bytes) 2024-09-21T23:17:00.604 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:00.600+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:00.617 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:00.617 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:00.617 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:00.627 INFO:teuthology.orchestra.run.smithi170.stdout:4064 write 0xe203000 thru 0xe2109ed (0xd9ee bytes) 2024-09-21T23:17:00.630 INFO:teuthology.orchestra.run.smithi170.stdout:4068 write 0x7336c00 thru 0x7344279 (0xd67a bytes) 2024-09-21T23:17:00.630 INFO:teuthology.orchestra.run.smithi170.stdout:4069 read 0x7314e00 thru 0x731fa0b (0xac0c bytes) 2024-09-21T23:17:00.630 INFO:teuthology.orchestra.run.smithi170.stdout:4070 read 0x4b82000 thru 0x4b8770e (0x570f bytes) 2024-09-21T23:17:00.633 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:00.649 INFO:teuthology.orchestra.run.smithi170.stdout:4074 trunc from 0xe210a00 to 0x3382200 2024-09-21T23:17:00.683 INFO:teuthology.orchestra.run.smithi170.stdout:4075 read 0x1616400 thru 0x1617a28 (0x1629 bytes) 2024-09-21T23:17:00.711 INFO:teuthology.orchestra.run.smithi170.stdout:4076 write 0x5f22200 thru 0x5f29f65 (0x7d66 bytes) 2024-09-21T23:17:00.718 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:00.716+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:00.748 INFO:teuthology.orchestra.run.smithi170.stdout:4077 write 0xad1bc00 thru 0xad20759 (0x4b5a bytes) 2024-09-21T23:17:00.751 INFO:teuthology.orchestra.run.smithi170.stdout:4078 read 0xab1400 thru 0xab7d94 (0x6995 bytes) 2024-09-21T23:17:00.752 INFO:teuthology.orchestra.run.smithi170.stdout:4082 read 0x3995800 thru 0x399f5b9 (0x9dba bytes) 2024-09-21T23:17:00.752 INFO:teuthology.orchestra.run.smithi170.stdout:4083 read 0x2d8c800 thru 0x2d9c02b (0xf82c bytes) 2024-09-21T23:17:00.753 INFO:teuthology.orchestra.run.smithi170.stdout:4084 write 0x8e02400 thru 0x8e0f319 (0xcf1a bytes) 2024-09-21T23:17:00.754 INFO:teuthology.orchestra.run.smithi170.stdout:4085 write 0x9678000 thru 0x967faa7 (0x7aa8 bytes) 2024-09-21T23:17:00.754 INFO:teuthology.orchestra.run.smithi170.stdout:4086 punch from 0x3258800 to 0x3260000, (0x7800 bytes) 2024-09-21T23:17:00.754 INFO:teuthology.orchestra.run.smithi170.stdout:4088 write 0x5f95400 thru 0x5f979dc (0x25dd bytes) 2024-09-21T23:17:00.760 INFO:teuthology.orchestra.run.smithi170.stdout:4089 trunc from 0xad20800 to 0x9684600 2024-09-21T23:17:00.791 INFO:teuthology.orchestra.run.smithi170.stdout:4091 trunc from 0x9684600 to 0x9b4fe00 2024-09-21T23:17:00.806 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:00.803+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:00.807 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.850582265, 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:00.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.813 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 85.850315280999993, 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:00.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237986736000003, 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237944036999998, 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.823 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:00.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:00.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237905417, 2024-09-21T23:17:00.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.831 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237873544999999, 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:00.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237840321, 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:00.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:00.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237805246000001, 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237760287999997, 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.844 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:00.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:00.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.237308220999999, 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.848 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:00.852 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.236593741999997, 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:00.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:00.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.856 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.236552672000002, 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.860 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.236095554000002, 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:00.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:00.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:00.864 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.235858874999998, 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.865 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.235443294, 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.234765633999999, 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:00.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:00.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:00.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:00.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:00.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:00.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:00.878 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.234274345000003, 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.234017446999999, 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:00.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:00.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26034 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7325696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.233554366, 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086107304999999995, 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26034 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571018+0000", 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018324500000000001 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571023+0000", 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9009999999999999e-06 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571048+0000", 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4454e-05 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571058+0000", 2024-09-21T23:17:00.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0145e-05 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656654+0000", 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085595579000000005 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656758+0000", 2024-09-21T23:17:00.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010446999999999999 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656942+0000", 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183511 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26035 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6836224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 44.233313981000002, 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086157396999999997, 2024-09-21T23:17:00.891 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26035 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571805+0000", 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000730326 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571812+0000", 2024-09-21T23:17:00.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571833+0000", 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1506999999999999e-05 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571846+0000", 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2330999999999999e-05 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657009+0000", 2024-09-21T23:17:00.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085162977000000001 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657091+0000", 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2514000000000003e-05 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657232+0000", 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140858 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:00.895 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:00.897 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4095 read 0x8f3e000 thru 0x8f48f21 (0xaf22 bytes) 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4097 write 0xbb400 thru 0xc8551 (0xd152 bytes) 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4098 write 0x8d0d600 thru 0x8d1446c (0x6e6d bytes) 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4100 write 0x866fa00 thru 0x867aa7c (0xb07d bytes) 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4102 read 0x97e6a00 thru 0x97ead27 (0x4328 bytes) 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4103 trunc from 0x9b4fe00 to 0x2fef000 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4104 read 0x1d5a400 thru 0x1d67ed7 (0xdad8 bytes) 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4105 read 0x2edee00 thru 0x2eecd28 (0xdf29 bytes) 2024-09-21T23:17:00.898 INFO:teuthology.orchestra.run.smithi170.stdout:4109 read 0xefac00 thru 0xf0121e (0x661f bytes) 2024-09-21T23:17:00.923 INFO:teuthology.orchestra.run.smithi170.stdout:4112 write 0xbd72c00 thru 0xbd79d7a (0x717b bytes) 2024-09-21T23:17:00.924 INFO:teuthology.orchestra.run.smithi170.stdout:4114 read 0x342ce00 thru 0x3437c48 (0xae49 bytes) 2024-09-21T23:17:00.926 INFO:teuthology.orchestra.run.smithi170.stdout:4115 write 0x90e6600 thru 0x90f31cc (0xcbcd bytes) 2024-09-21T23:17:00.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:00.942 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:00.940+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:00.951 INFO:teuthology.orchestra.run.smithi170.stdout:4118 trunc from 0xbd79e00 to 0x1091a00 2024-09-21T23:17:00.987 INFO:teuthology.orchestra.run.smithi170.stdout:4120 trunc from 0x1091a00 to 0x1b2c200 2024-09-21T23:17:01.005 INFO:teuthology.orchestra.run.smithi170.stdout:4123 read 0x902400 thru 0x903707 (0x1308 bytes) 2024-09-21T23:17:01.008 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:01.007+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:01.031 INFO:teuthology.orchestra.run.smithi170.stdout:4124 write 0x421f200 thru 0x4222522 (0x3323 bytes) 2024-09-21T23:17:01.058 INFO:teuthology.orchestra.run.smithi170.stdout:4125 trunc from 0x4222600 to 0xe2ee400 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5652.0:1638 3.1c 3:3ddd995c:::rbd_header.10f1a15a782f:head [call rbd.parent_get in=13b,call rbd.parent_overlap_get in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:01.082174+0000", 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00029206000000000002, 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033126199999999997, 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:01.083 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5652", 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2970316875", 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1638 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082174+0000", 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082174+0000", 2024-09-21T23:17:01.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082176+0000", 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.871e-06 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082186+0000", 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9170000000000001e-06 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082207+0000", 2024-09-21T23:17:01.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1070999999999999e-05 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082226+0000", 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8226000000000001e-05 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082306+0000", 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0576999999999998e-05 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:01.082402+0000", 2024-09-21T23:17:01.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6209e-05 2024-09-21T23:17:01.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:01.087 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:01.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:01.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:01.087 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:17:01.087 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:17:01.087 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:01.088 INFO:teuthology.orchestra.run.smithi170.stdout:4126 write 0x3215a00 thru 0x3218e57 (0x3458 bytes) 2024-09-21T23:17:01.088 INFO:teuthology.orchestra.run.smithi170.stdout:4127 write 0xa301e00 thru 0xa3054c0 (0x36c1 bytes) 2024-09-21T23:17:01.088 INFO:teuthology.orchestra.run.smithi170.stdout:4130 write 0x93dce00 thru 0x93ecdcb (0xffcc bytes) 2024-09-21T23:17:01.088 INFO:teuthology.orchestra.run.smithi170.stdout:4133 write 0xae31c00 thru 0xae3b19d (0x959e bytes) 2024-09-21T23:17:01.088 INFO:teuthology.orchestra.run.smithi170.stdout:4135 punch from 0x6a01600 to 0x6a08600, (0x7000 bytes) 2024-09-21T23:17:01.089 INFO:teuthology.orchestra.run.smithi170.stdout:4136 read 0x6734a00 thru 0x6739f8d (0x558e bytes) 2024-09-21T23:17:01.089 INFO:teuthology.orchestra.run.smithi170.stdout:4138 punch from 0xb294600 to 0xb2a2000, (0xda00 bytes) 2024-09-21T23:17:01.089 INFO:teuthology.orchestra.run.smithi170.stdout:4140 read 0x79f8600 thru 0x7a00026 (0x7a27 bytes) 2024-09-21T23:17:01.092 INFO:teuthology.orchestra.run.smithi170.stdout:4144 write 0x1b1e000 thru 0x1b283ae (0xa3af bytes) 2024-09-21T23:17:01.092 INFO:teuthology.orchestra.run.smithi170.stdout:4150 write 0x142e000 thru 0x1434ae7 (0x6ae8 bytes) 2024-09-21T23:17:01.093 INFO:teuthology.orchestra.run.smithi170.stdout:4152 read 0x2dca200 thru 0x2dd70a2 (0xcea3 bytes) 2024-09-21T23:17:01.098 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:01.099 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:01.100 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:01.106 INFO:teuthology.orchestra.run.smithi170.stdout:4153 trunc from 0xe2ee400 to 0x743ea00 2024-09-21T23:17:01.116 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:01.116 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:01.133 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:01.133 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:01.145 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:01.144+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:01.151 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:01.152 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:01.155 INFO:teuthology.orchestra.run.smithi170.stdout:4155 write 0xe3be400 thru 0xe3c56dc (0x72dd bytes) 2024-09-21T23:17:01.156 INFO:teuthology.orchestra.run.smithi170.stdout:4159 punch from 0xc6d4600 to 0xc6db800, (0x7200 bytes) 2024-09-21T23:17:01.157 INFO:teuthology.orchestra.run.smithi170.stdout:4161 write 0x4dbde00 thru 0x4dcdc33 (0xfe34 bytes) 2024-09-21T23:17:01.157 INFO:teuthology.orchestra.run.smithi170.stdout:4162 read 0x71cac00 thru 0x71d2479 (0x787a bytes) 2024-09-21T23:17:01.158 INFO:teuthology.orchestra.run.smithi170.stdout:4163 read 0x804b400 thru 0x8055681 (0xa282 bytes) 2024-09-21T23:17:01.158 INFO:teuthology.orchestra.run.smithi170.stdout:4164 write 0xdbd7000 thru 0xdbde084 (0x7085 bytes) 2024-09-21T23:17:01.158 INFO:teuthology.orchestra.run.smithi170.stdout:4168 write 0x7f98200 thru 0x7fa2eb0 (0xacb1 bytes) 2024-09-21T23:17:01.158 INFO:teuthology.orchestra.run.smithi170.stdout:4169 read 0xacbb800 thru 0xacc3f5d (0x875e bytes) 2024-09-21T23:17:01.168 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:01.169 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:01.177 INFO:teuthology.orchestra.run.smithi170.stdout:4171 trunc from 0xe3c5800 to 0x3792e00 2024-09-21T23:17:01.188 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:01.188 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:01.202 INFO:teuthology.orchestra.run.smithi170.stdout:4172 punch from 0x2f36a00 to 0x2f38600, (0x1c00 bytes) 2024-09-21T23:17:01.203 INFO:teuthology.orchestra.run.smithi170.stdout:4173 read 0x12ae00 thru 0x136cb4 (0xbeb5 bytes) 2024-09-21T23:17:01.227 INFO:teuthology.orchestra.run.smithi170.stdout:4174 write 0x78fdc00 thru 0x79093d8 (0xb7d9 bytes) 2024-09-21T23:17:01.234 INFO:teuthology.orchestra.run.smithi170.stdout:4175 trunc from 0x7909400 to 0x4872600 2024-09-21T23:17:01.239 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:01.239 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:01.251 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:01.251+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:01.255 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:01.257 INFO:teuthology.orchestra.run.smithi170.stdout:4177 punch from 0x3b1c200 to 0x3b28e00, (0xcc00 bytes) 2024-09-21T23:17:01.258 INFO:teuthology.orchestra.run.smithi170.stdout:4180 read 0x215ea00 thru 0x216d7b3 (0xedb4 bytes) 2024-09-21T23:17:01.262 INFO:teuthology.orchestra.run.smithi170.stdout:4181 read 0x43e4400 thru 0x43e8603 (0x4204 bytes) 2024-09-21T23:17:01.294 INFO:teuthology.orchestra.run.smithi170.stdout:4183 write 0xc8c9000 thru 0xc8d4ff1 (0xbff2 bytes) 2024-09-21T23:17:01.295 INFO:teuthology.orchestra.run.smithi170.stdout:4184 read 0x88e2400 thru 0x88eea70 (0xc671 bytes) 2024-09-21T23:17:01.310 INFO:teuthology.orchestra.run.smithi170.stdout:4185 trunc from 0xc8d5000 to 0x50ede00 2024-09-21T23:17:01.313 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:01.313 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:01.313 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T23:17:01.313 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:17:01.313 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:17:01.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T23:17:01.341 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:01.340+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:01.349 INFO:teuthology.orchestra.run.smithi170.stdout:4186 trunc from 0x50ede00 to 0x30f8e00 2024-09-21T23:17:01.373 INFO:teuthology.orchestra.run.smithi170.stdout:4187 write 0xde1400 thru 0xde9758 (0x8359 bytes) 2024-09-21T23:17:01.375 INFO:teuthology.orchestra.run.smithi170.stdout:4189 read 0x18ef600 thru 0x18fd974 (0xe375 bytes) 2024-09-21T23:17:01.411 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:01.408+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:01.413 INFO:teuthology.orchestra.run.smithi170.stdout:4190 write 0x9706800 thru 0x9710a73 (0xa274 bytes) 2024-09-21T23:17:01.414 INFO:teuthology.orchestra.run.smithi170.stdout:4192 write 0x296b000 thru 0x297774b (0xc74c bytes) 2024-09-21T23:17:01.432 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:01.432 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:01.432 INFO:teuthology.orchestra.run.smithi170.stdout:4196 trunc from 0x9710c00 to 0x1569c00 2024-09-21T23:17:01.447 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:01.450 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:01.450 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:17:01.450 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:01.488 INFO:teuthology.orchestra.run.smithi170.stdout:4197 trunc from 0x1569c00 to 0x9f14e00 2024-09-21T23:17:01.512 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:01.508+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:01.515 INFO:teuthology.orchestra.run.smithi170.stdout:4198 trunc from 0x9f14e00 to 0x4afbe00 2024-09-21T23:17:01.538 INFO:teuthology.orchestra.run.smithi170.stdout:4201 write 0x46e3c00 thru 0x46e7c5b (0x405c bytes) 2024-09-21T23:17:01.541 INFO:teuthology.orchestra.run.smithi170.stdout:4202 trunc from 0x4afbe00 to 0x39a9000 2024-09-21T23:17:01.576 INFO:teuthology.orchestra.run.smithi170.stdout:4203 write 0x1291000 thru 0x12988cf (0x78d0 bytes) 2024-09-21T23:17:01.579 INFO:teuthology.orchestra.run.smithi170.stdout:4204 read 0x681400 thru 0x68a34d (0x8f4e bytes) 2024-09-21T23:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:01.591 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:01.601 INFO:teuthology.orchestra.run.smithi170.stdout:4205 write 0x6ad6a00 thru 0x6adb42e (0x4a2f bytes) 2024-09-21T23:17:01.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:01.613 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:01.608+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:01.626 INFO:teuthology.orchestra.run.smithi170.stdout:4206 write 0x81ca400 thru 0x81d74a2 (0xd0a3 bytes) 2024-09-21T23:17:01.639 INFO:teuthology.orchestra.run.smithi170.stdout:4207 trunc from 0x81d7600 to 0x301aa00 2024-09-21T23:17:01.669 INFO:teuthology.orchestra.run.smithi170.stdout:4208 punch from 0x20f9600 to 0x20fac00, (0x1600 bytes) 2024-09-21T23:17:01.671 INFO:teuthology.orchestra.run.smithi170.stdout:4209 clone 26 order 18 su 32768 sc 11 2024-09-21T23:17:01.714 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:01.712+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:01.736 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:01.736 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:01.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:01.815 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:01.812+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:01.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:17:01.877 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:01.877 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:01.892 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:01.939 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:01.939+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:02.017 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:02.015+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:02.065 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:02.065 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:02.080 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:02.118 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:02.116+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:02.220 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:02.216+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:02.250 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:02.250 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:02.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:02.320 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:02.319+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:02.397 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:02.397 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:02.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:02.455 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:02.455+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:02.522 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:02.520+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:02.540 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-21T23:17:02.591 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:02.591 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:02.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:02.623 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:02.620+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:02.638 INFO:teuthology.orchestra.run.smithi170.stdout:4214 punch from 0x6bf600 to 0x6c8800, (0x9200 bytes) 2024-09-21T23:17:02.723 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:02.723+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:02.729 INFO:teuthology.orchestra.run.smithi170.stdout:4215 write 0xb6f5400 thru 0xb6f9686 (0x4287 bytes) 2024-09-21T23:17:02.734 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:02.734 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:02.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:02.789 INFO:teuthology.orchestra.run.smithi170.stdout:4216 write 0xed96600 thru 0xed9aa17 (0x4418 bytes) 2024-09-21T23:17:02.792 INFO:teuthology.orchestra.run.smithi170.stdout:4217 punch from 0x948a000 to 0x948b000, (0x1000 bytes) 2024-09-21T23:17:02.792 INFO:teuthology.orchestra.run.smithi170.stdout:4218 read 0x716b600 thru 0x717aecc (0xf8cd bytes) 2024-09-21T23:17:02.793 INFO:teuthology.orchestra.run.smithi170.stdout:4219 write 0x642ba00 thru 0x6432e68 (0x7469 bytes) 2024-09-21T23:17:02.793 INFO:teuthology.orchestra.run.smithi170.stdout:4220 read 0xbe08c00 thru 0xbe0bd58 (0x3159 bytes) 2024-09-21T23:17:02.808 INFO:teuthology.orchestra.run.smithi170.stdout:4222 trunc from 0xed9ac00 to 0x76c1000 2024-09-21T23:17:02.813 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:17:02.824 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:02.823+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:02.867 INFO:teuthology.orchestra.run.smithi170.stdout:4224 punch from 0x62b0600 to 0x62b5a00, (0x5400 bytes) 2024-09-21T23:17:02.869 INFO:teuthology.orchestra.run.smithi170.stdout:4226 read 0x2093600 thru 0x209cffd (0x99fe bytes) 2024-09-21T23:17:02.873 INFO:teuthology.orchestra.run.smithi170.stdout:4227 read 0x307a200 thru 0x3081126 (0x6f27 bytes) 2024-09-21T23:17:02.874 INFO:teuthology.orchestra.run.smithi170.stdout:4229 write 0x13c3200 thru 0x13c9fce (0x6dcf bytes) 2024-09-21T23:17:02.877 INFO:teuthology.orchestra.run.smithi170.stdout:4231 read 0x92b800 thru 0x9314d9 (0x5cda bytes) 2024-09-21T23:17:02.924 INFO:teuthology.orchestra.run.smithi170.stdout:4234 write 0xc166e00 thru 0xc16d496 (0x6697 bytes) 2024-09-21T23:17:02.924 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:02.925 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:02.925 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:02.923+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:02.926 INFO:teuthology.orchestra.run.smithi170.stdout:4236 read 0x27d0400 thru 0x27d8d1b (0x891c bytes) 2024-09-21T23:17:02.927 INFO:teuthology.orchestra.run.smithi170.stdout:4240 write 0x2120200 thru 0x2121ef5 (0x1cf6 bytes) 2024-09-21T23:17:02.930 INFO:teuthology.orchestra.run.smithi170.stdout:4242 read 0x5a9400 thru 0x5b91aa (0xfdab bytes) 2024-09-21T23:17:02.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:02.990 INFO:teuthology.orchestra.run.smithi170.stdout:4243 trunc from 0xc16d600 to 0x731000 2024-09-21T23:17:03.025 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:03.020+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:03.078 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:03.078 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:03.088 INFO:teuthology.orchestra.run.smithi170.stdout:4244 write 0x4998400 thru 0x499d096 (0x4c97 bytes) 2024-09-21T23:17:03.089 INFO:teuthology.orchestra.run.smithi170.stdout:4246 read 0x1539a00 thru 0x153e8bd (0x4ebe bytes) 2024-09-21T23:17:03.089 INFO:teuthology.orchestra.run.smithi170.stdout:4247 read 0x1be5c00 thru 0x1bf4b43 (0xef44 bytes) 2024-09-21T23:17:03.090 INFO:teuthology.orchestra.run.smithi170.stdout:4248 write 0x1dd5200 thru 0x1de439c (0xf19d bytes) 2024-09-21T23:17:03.090 INFO:teuthology.orchestra.run.smithi170.stdout:4251 punch from 0x3667600 to 0x366de00, (0x6800 bytes) 2024-09-21T23:17:03.093 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:03.094 INFO:teuthology.orchestra.run.smithi170.stdout:4255 trunc from 0x499d200 to 0x3bbe400 2024-09-21T23:17:03.134 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:03.132+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:03.154 INFO:teuthology.orchestra.run.smithi170.stdout:4257 write 0x4b79000 thru 0x4b84eb3 (0xbeb4 bytes) 2024-09-21T23:17:03.165 INFO:teuthology.orchestra.run.smithi170.stdout:4259 trunc from 0x4b85000 to 0x9db4a00 2024-09-21T23:17:03.188 INFO:teuthology.orchestra.run.smithi170.stdout:4260 write 0x285d000 thru 0x286bc0f (0xec10 bytes) 2024-09-21T23:17:03.190 INFO:teuthology.orchestra.run.smithi170.stdout:4263 read 0x2c92200 thru 0x2c9416a (0x1f6b bytes) 2024-09-21T23:17:03.190 INFO:teuthology.orchestra.run.smithi170.stdout:4267 read 0x290e600 thru 0x2917178 (0x8b79 bytes) 2024-09-21T23:17:03.190 INFO:teuthology.orchestra.run.smithi170.stdout:4268 read 0x8e06400 thru 0x8e09d52 (0x3953 bytes) 2024-09-21T23:17:03.190 INFO:teuthology.orchestra.run.smithi170.stdout:4271 write 0xa03600 thru 0xa0a2dc (0x6cdd bytes) 2024-09-21T23:17:03.191 INFO:teuthology.orchestra.run.smithi170.stdout:4272 write 0x724c800 thru 0x7253052 (0x6853 bytes) 2024-09-21T23:17:03.191 INFO:teuthology.orchestra.run.smithi170.stdout:4276 write 0x78b7a00 thru 0x78b88ab (0xeac bytes) 2024-09-21T23:17:03.199 INFO:teuthology.orchestra.run.smithi170.stdout:4278 trunc from 0x9db4a00 to 0x6003200 2024-09-21T23:17:03.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:03.239 INFO:teuthology.orchestra.run.smithi170.stdout:4279 write 0x228b000 thru 0x2291c1d (0x6c1e bytes) 2024-09-21T23:17:03.251 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:03.251+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:03.264 INFO:teuthology.orchestra.run.smithi170.stdout:4280 trunc from 0x6003200 to 0xeafac00 2024-09-21T23:17:03.276 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:03.277 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:03.287 INFO:teuthology.orchestra.run.smithi170.stdout:4282 punch from 0x6da4e00 to 0x6daf200, (0xa400 bytes) 2024-09-21T23:17:03.289 INFO:teuthology.orchestra.run.smithi170.stdout:4284 read 0x222a000 thru 0x222ad5e (0xd5f bytes) 2024-09-21T23:17:03.289 INFO:teuthology.orchestra.run.smithi170.stdout:4285 punch from 0x1cb1000 to 0x1cbae00, (0x9e00 bytes) 2024-09-21T23:17:03.290 INFO:teuthology.orchestra.run.smithi170.stdout:4286 write 0xe81000 thru 0xe87726 (0x6727 bytes) 2024-09-21T23:17:03.290 INFO:teuthology.orchestra.run.smithi170.stdout:4288 write 0xb25b400 thru 0xb25c795 (0x1396 bytes) 2024-09-21T23:17:03.290 INFO:teuthology.orchestra.run.smithi170.stdout:4291 punch from 0xcc22e00 to 0xcc2f800, (0xca00 bytes) 2024-09-21T23:17:03.291 INFO:teuthology.orchestra.run.smithi170.stdout:4292 read 0xaa30e00 thru 0xaa38f34 (0x8135 bytes) 2024-09-21T23:17:03.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:03.305 INFO:teuthology.orchestra.run.smithi170.stdout:4293 trunc from 0xeafac00 to 0x7795a00 2024-09-21T23:17:03.326 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:03.323+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:03.331 INFO:teuthology.orchestra.run.smithi170.stdout:4296 write 0x64d7800 thru 0x64deb74 (0x7375 bytes) 2024-09-21T23:17:03.332 INFO:teuthology.orchestra.run.smithi170.stdout:4297 read 0x5344400 thru 0x534fd12 (0xb913 bytes) 2024-09-21T23:17:03.332 INFO:teuthology.orchestra.run.smithi170.stdout:4298 write 0x385f600 thru 0x386eab1 (0xf4b2 bytes) 2024-09-21T23:17:03.361 INFO:teuthology.orchestra.run.smithi170.stdout:4299 write 0xcbe5200 thru 0xcbe6647 (0x1448 bytes) 2024-09-21T23:17:03.363 INFO:teuthology.orchestra.run.smithi170.stdout:4301 read 0x999da00 thru 0x999ef21 (0x1522 bytes) 2024-09-21T23:17:03.363 INFO:teuthology.orchestra.run.smithi170.stdout:4302 read 0xb596e00 thru 0xb59a0da (0x32db bytes) 2024-09-21T23:17:03.363 INFO:teuthology.orchestra.run.smithi170.stdout:4303 write 0x799d200 thru 0x79a5bda (0x89db bytes) 2024-09-21T23:17:03.363 INFO:teuthology.orchestra.run.smithi170.stdout:4304 read 0x8a80800 thru 0x8a8cc09 (0xc40a bytes) 2024-09-21T23:17:03.369 INFO:teuthology.orchestra.run.smithi170.stdout:4308 trunc from 0xcbe6800 to 0x988e000 2024-09-21T23:17:03.373 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:03.373 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:03.391 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:03.396 INFO:teuthology.orchestra.run.smithi170.stdout:4309 punch from 0x587b000 to 0x5881a00, (0x6a00 bytes) 2024-09-21T23:17:03.397 INFO:teuthology.orchestra.run.smithi170.stdout:4312 read 0x950600 thru 0x956140 (0x5b41 bytes) 2024-09-21T23:17:03.408 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:03.421 INFO:teuthology.orchestra.run.smithi170.stdout:4313 write 0xe26aa00 thru 0xe26d207 (0x2808 bytes) 2024-09-21T23:17:03.422 INFO:teuthology.orchestra.run.smithi170.stdout:4315 write 0x57f2c00 thru 0x580072c (0xdb2d bytes) 2024-09-21T23:17:03.423 INFO:teuthology.orchestra.run.smithi170.stdout:4316 write 0x7dfc000 thru 0x7e089ae (0xc9af bytes) 2024-09-21T23:17:03.423 INFO:teuthology.orchestra.run.smithi170.stdout:4318 read 0x1993e00 thru 0x19a1e8b (0xe08c bytes) 2024-09-21T23:17:03.424 INFO:teuthology.orchestra.run.smithi170.stdout:4320 read 0xa71ea00 thru 0xa723684 (0x4c85 bytes) 2024-09-21T23:17:03.424 INFO:teuthology.orchestra.run.smithi170.stdout:4321 read 0x10f7000 thru 0x10f9ab2 (0x2ab3 bytes) 2024-09-21T23:17:03.424 INFO:teuthology.orchestra.run.smithi170.stdout:4324 write 0x824fa00 thru 0x824faae (0xaf bytes) 2024-09-21T23:17:03.424 INFO:teuthology.orchestra.run.smithi170.stdout:4327 read 0x9cb3200 thru 0x9cb63e9 (0x31ea bytes) 2024-09-21T23:17:03.425 INFO:teuthology.orchestra.run.smithi170.stdout:4328 read 0x3824600 thru 0x382c850 (0x8251 bytes) 2024-09-21T23:17:03.425 INFO:teuthology.orchestra.run.smithi170.stdout:4329 read 0x9cae800 thru 0x9cb1942 (0x3143 bytes) 2024-09-21T23:17:03.425 INFO:teuthology.orchestra.run.smithi170.stdout:4330 write 0x6eee600 thru 0x6ef8110 (0x9b11 bytes) 2024-09-21T23:17:03.425 INFO:teuthology.orchestra.run.smithi170.stdout:4334 write 0x549a000 thru 0x54a9cd7 (0xfcd8 bytes) 2024-09-21T23:17:03.427 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:03.424+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:03.428 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:03.428 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:03.438 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 137.21641395099999, 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.704008668, 2024-09-21T23:17:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.70394376499999, 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:03.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:03.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:03.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.703797326, 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.458 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:03.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:03.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:03.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:03.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.703728211, 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:03.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.70311588600001, 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:03.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:03.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.702911481, 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:03.478 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:03.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:03.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.70286869500001, 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:03.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:03.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.311910407, 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:03.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:03.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:03.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.31182389700001, 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:03.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:03.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:03.506 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.311747008, 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:03.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:03.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.31154216799999, 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.514 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:03.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:03.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:03.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.31145388600001, 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.521 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:03.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:03.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:03.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.311323058, 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.528 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:03.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:03.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:03.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.311277405, 2024-09-21T23:17:03.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:03.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:03.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:03.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:03.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.31123897800001, 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:03.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:03.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.31110951599999, 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:03.550 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:03.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:03.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.31108787400001, 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:03.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:03.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.564 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.31106783000001, 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.565 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:03.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:03.571 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:03.572 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 136.310983838, 2024-09-21T23:17:03.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:03.572 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.572 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.572 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:03.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:03.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:03.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:03.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.580 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:03.582 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:03.583 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:03.584 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:03.584 INFO:teuthology.orchestra.run.smithi170.stdout:4335 trunc from 0xe26d400 to 0x4d76a00 2024-09-21T23:17:03.584 INFO:teuthology.orchestra.run.smithi170.stdout:4336 trunc from 0x4d76a00 to 0xc11d800 2024-09-21T23:17:03.584 INFO:teuthology.orchestra.run.smithi170.stdout:4338 trunc from 0xc11d800 to 0x8096200 2024-09-21T23:17:03.585 INFO:teuthology.orchestra.run.smithi170.stdout:4342 read 0x2b57600 thru 0x2b62127 (0xab28 bytes) 2024-09-21T23:17:03.585 INFO:teuthology.orchestra.run.smithi170.stdout:4346 read 0x5664400 thru 0x566bfa1 (0x7ba2 bytes) 2024-09-21T23:17:03.585 INFO:teuthology.orchestra.run.smithi170.stdout:4347 trunc from 0x8096200 to 0x165ea00 2024-09-21T23:17:03.585 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:03.585 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:03.580+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:03.586 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:03.641 INFO:teuthology.orchestra.run.smithi170.stdout:4348 write 0x255dc00 thru 0x2569e3f (0xc240 bytes) 2024-09-21T23:17:03.642 INFO:teuthology.orchestra.run.smithi170.stdout:4349 punch from 0x1b79600 to 0x1b86000, (0xca00 bytes) 2024-09-21T23:17:03.643 INFO:teuthology.orchestra.run.smithi170.stdout:4350 read 0xa5d400 thru 0xa623b9 (0x4fba bytes) 2024-09-21T23:17:03.649 INFO:teuthology.orchestra.run.smithi170.stdout:4352 punch from 0x185f600 to 0x186a800, (0xb200 bytes) 2024-09-21T23:17:03.650 INFO:teuthology.orchestra.run.smithi170.stdout:4356 read 0x2533400 thru 0x253d6c0 (0xa2c1 bytes) 2024-09-21T23:17:03.650 INFO:teuthology.orchestra.run.smithi170.stdout:4362 read 0x1f3bc00 thru 0x1f3bf2a (0x32b bytes) 2024-09-21T23:17:03.656 INFO:teuthology.orchestra.run.smithi170.stdout:4364 trunc from 0x256a000 to 0x4eb8400 2024-09-21T23:17:03.672 INFO:teuthology.orchestra.run.smithi170.stdout:4365 read 0x2880e00 thru 0x28828a3 (0x1aa4 bytes) 2024-09-21T23:17:03.674 INFO:teuthology.orchestra.run.smithi170.stdout:4367 write 0x2a0f400 thru 0x2a1b8fb (0xc4fc bytes) 2024-09-21T23:17:03.674 INFO:teuthology.orchestra.run.smithi170.stdout:4368 punch from 0x1cb1800 to 0x1cb7e00, (0x6600 bytes) 2024-09-21T23:17:03.675 INFO:teuthology.orchestra.run.smithi170.stdout:4372 read 0x1e5ba00 thru 0x1e6a568 (0xeb69 bytes) 2024-09-21T23:17:03.675 INFO:teuthology.orchestra.run.smithi170.stdout:4373 write 0x2dbd400 thru 0x2dbeaba (0x16bb bytes) 2024-09-21T23:17:03.675 INFO:teuthology.orchestra.run.smithi170.stdout:4374 read 0x71e600 thru 0x725e0a (0x780b bytes) 2024-09-21T23:17:03.676 INFO:teuthology.orchestra.run.smithi170.stdout:4375 read 0x2b7cc00 thru 0x2b80ee4 (0x42e5 bytes) 2024-09-21T23:17:03.676 INFO:teuthology.orchestra.run.smithi170.stdout:4376 read 0x1c1be00 thru 0x1c27f62 (0xc163 bytes) 2024-09-21T23:17:03.676 INFO:teuthology.orchestra.run.smithi170.stdout:4378 read 0x1d83e00 thru 0x1d85004 (0x1205 bytes) 2024-09-21T23:17:03.677 INFO:teuthology.orchestra.run.smithi170.stdout:4379 write 0x879000 thru 0x888db4 (0xfdb5 bytes) 2024-09-21T23:17:03.677 INFO:teuthology.orchestra.run.smithi170.stdout:4380 read 0x33fe200 thru 0x340a15f (0xbf60 bytes) 2024-09-21T23:17:03.677 INFO:teuthology.orchestra.run.smithi170.stdout:4382 write 0x1909800 thru 0x190c492 (0x2c93 bytes) 2024-09-21T23:17:03.678 INFO:teuthology.orchestra.run.smithi170.stdout:4387 write 0x3ae8c00 thru 0x3af1181 (0x8582 bytes) 2024-09-21T23:17:03.683 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:03.680+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:03.691 INFO:teuthology.orchestra.run.smithi170.stdout:4389 trunc from 0x4eb8400 to 0xcbb4c00 2024-09-21T23:17:03.741 INFO:teuthology.orchestra.run.smithi170.stdout:4391 write 0xe7d6600 thru 0xe7dfda7 (0x97a8 bytes) 2024-09-21T23:17:03.742 INFO:teuthology.orchestra.run.smithi170.stdout:4392 read 0x5032e00 thru 0x503a7fe (0x79ff bytes) 2024-09-21T23:17:03.742 INFO:teuthology.orchestra.run.smithi170.stdout:4394 write 0xb69fe00 thru 0xb6ae36f (0xe570 bytes) 2024-09-21T23:17:03.760 INFO:teuthology.orchestra.run.smithi170.stdout:4396 trunc from 0xe7dfe00 to 0x6d58800 2024-09-21T23:17:03.771 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5694.0:2313 3.a 3:5231034a:::rbd_data.10f1374a4ee4.00000000000002d8:head [set-alloc-hint object_size 262144 write_size 262144,write 130560~512 in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:03.768928+0000", 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0025893029999999998, 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0026122229999999999, 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5694", 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3277696404", 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 2313 2024-09-21T23:17:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.768928+0000", 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.768928+0000", 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.768930+0000", 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7689999999999999e-06 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.768937+0000", 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8339999999999999e-06 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.768955+0000", 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7527e-05 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.768963+0000", 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8210000000000002e-06 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.769050+0000", 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7677999999999996e-05 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.769137+0000", 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6382000000000001e-05 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.769267+0000", 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012979199999999999 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:03.770138+0000", 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087100000000000003 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:17:03.776 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:03.783 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:03.780+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:03.788 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:03.812 INFO:teuthology.orchestra.run.smithi170.stdout:4398 trunc from 0x6d58800 to 0x10b6400 2024-09-21T23:17:03.847 INFO:teuthology.orchestra.run.smithi170.stdout:4400 punch from 0x25cc00 to 0x25fc00, (0x3000 bytes) 2024-09-21T23:17:03.873 INFO:teuthology.orchestra.run.smithi170.stdout:4402 write 0x15ac000 thru 0x15b9995 (0xd996 bytes) 2024-09-21T23:17:03.885 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:03.883+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:03.909 INFO:teuthology.orchestra.run.smithi170.stdout:4403 write 0x6a94600 thru 0x6a96fde (0x29df bytes) 2024-09-21T23:17:03.922 INFO:teuthology.orchestra.run.smithi170.stdout:4404 trunc from 0x6a97000 to 0xe8dc00 2024-09-21T23:17:03.962 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:03.962 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:03.962 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:03.962 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:03.962 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:03.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:03.979 INFO:teuthology.orchestra.run.smithi170.stdout:4405 trunc from 0xe8dc00 to 0xab67400 2024-09-21T23:17:03.994 INFO:teuthology.orchestra.run.smithi170.stdout:4406 read 0x3884400 thru 0x38932af (0xeeb0 bytes) 2024-09-21T23:17:03.999 INFO:teuthology.orchestra.run.smithi170.stdout:4407 trunc from 0xab67400 to 0xc7b8200 2024-09-21T23:17:04.012 INFO:teuthology.orchestra.run.smithi170.stdout:4414 punch from 0x6878000 to 0x6881800, (0x9800 bytes) 2024-09-21T23:17:04.014 INFO:teuthology.orchestra.run.smithi170.stdout:4415 write 0x201d200 thru 0x20214f2 (0x42f3 bytes) 2024-09-21T23:17:04.014 INFO:teuthology.orchestra.run.smithi170.stdout:4418 write 0x8c12c00 thru 0x8c160cd (0x34ce bytes) 2024-09-21T23:17:04.021 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:04.020+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:04.022 INFO:teuthology.orchestra.run.smithi170.stdout:4419 trunc from 0xc7b8200 to 0x88d0c00 2024-09-21T23:17:04.073 INFO:teuthology.orchestra.run.smithi170.stdout:4420 write 0xb53cc00 thru 0xb542ee7 (0x62e8 bytes) 2024-09-21T23:17:04.075 INFO:teuthology.orchestra.run.smithi170.stdout:4421 write 0xa88ac00 thru 0xa88ec8c (0x408d bytes) 2024-09-21T23:17:04.075 INFO:teuthology.orchestra.run.smithi170.stdout:4423 punch from 0xa99c600 to 0xa99ee00, (0x2800 bytes) 2024-09-21T23:17:04.086 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:04.084+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:04.099 INFO:teuthology.orchestra.run.smithi170.stdout:4426 write 0xe54c400 thru 0xe5577da (0xb3db bytes) 2024-09-21T23:17:04.100 INFO:teuthology.orchestra.run.smithi170.stdout:4427 write 0x717c400 thru 0x71866bb (0xa2bc bytes) 2024-09-21T23:17:04.101 INFO:teuthology.orchestra.run.smithi170.stdout:4428 trunc from 0xe557800 to 0xec6e400 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:04.116 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 89.160723900999997, 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:04.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:04.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:04.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 89.160456917000005, 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:04.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:04.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:04.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:04.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.179 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.548128372000001, 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:04.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.548085673000003, 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.184 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:04.188 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.548047052999998, 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:04.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.193 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.193 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.548015180999997, 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.195 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:04.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.547981956999998, 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.199 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:04.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.203 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.547946881999998, 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:04.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:04.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:04.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.547901924000001, 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:04.212 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.547449856999997, 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:04.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:04.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:04.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.546735378000001, 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.217 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.221 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.546694307999999, 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.222 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:04.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.546237189999999, 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:04.226 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:04.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:04.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.546000511000003, 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:04.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:04.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.545584929999997, 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:04.235 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:04.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.544907270000003, 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:04.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:04.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.244 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.544415981, 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.248 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.544159082999997, 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:04.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26034 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7325696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.543696001999997, 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086107304999999995, 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.253 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26034 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570834+0000", 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571018+0000", 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018324500000000001 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571023+0000", 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9009999999999999e-06 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571048+0000", 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4454e-05 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571058+0000", 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0145e-05 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656654+0000", 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085595579000000005 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656758+0000", 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010446999999999999 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656942+0000", 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183511 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.257 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26035 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6836224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.543455616999999, 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086157396999999997, 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26035 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571075+0000", 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571805+0000", 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000730326 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571812+0000", 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571833+0000", 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1506999999999999e-05 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.571846+0000", 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2330999999999999e-05 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657009+0000", 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085162977000000001 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657091+0000", 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2514000000000003e-05 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.657232+0000", 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140858 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.261 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.262 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.262 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:04.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:04.264 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:04.260+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4429 write 0xc6c2400 thru 0xc6cbbb8 (0x97b9 bytes) 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4430 read 0x9e400 thru 0xa313f (0x4d40 bytes) 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4431 punch from 0xbc31800 to 0xbc36e00, (0x5600 bytes) 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4432 trunc from 0xec6e400 to 0x1c1f400 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4433 punch from 0x1c0d800 to 0x1c10e00, (0x3600 bytes) 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4434 write 0xe730600 thru 0xe73b99c (0xb39d bytes) 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4436 write 0x7be3600 thru 0x7be7af2 (0x44f3 bytes) 2024-09-21T23:17:04.265 INFO:teuthology.orchestra.run.smithi170.stdout:4437 trunc from 0xe73ba00 to 0x9acac00 2024-09-21T23:17:04.297 INFO:teuthology.orchestra.run.smithi170.stdout:4438 punch from 0x5f5d600 to 0x5f6cc00, (0xf600 bytes) 2024-09-21T23:17:04.302 INFO:teuthology.orchestra.run.smithi170.stdout:4440 write 0x294aa00 thru 0x29539af (0x8fb0 bytes) 2024-09-21T23:17:04.302 INFO:teuthology.orchestra.run.smithi170.stdout:4442 punch from 0xbec200 to 0xbf2c00, (0x6a00 bytes) 2024-09-21T23:17:04.302 INFO:teuthology.orchestra.run.smithi170.stdout:4444 write 0x3244400 thru 0x324bfa9 (0x7baa bytes) 2024-09-21T23:17:04.336 INFO:teuthology.orchestra.run.smithi170.stdout:4445 write 0xa01e200 thru 0xa02dd04 (0xfb05 bytes) 2024-09-21T23:17:04.341 INFO:teuthology.orchestra.run.smithi170.stdout:4446 trunc from 0xa02de00 to 0xb139000 2024-09-21T23:17:04.364 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:04.360+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:04.367 INFO:teuthology.orchestra.run.smithi170.stdout:4447 trunc from 0xb139000 to 0x91d9c00 2024-09-21T23:17:04.406 INFO:teuthology.orchestra.run.smithi170.stdout:4448 trunc from 0x91d9c00 to 0xe885e00 2024-09-21T23:17:04.421 INFO:teuthology.orchestra.run.smithi170.stdout:4449 read 0xcc74c00 thru 0xcc823d2 (0xd7d3 bytes) 2024-09-21T23:17:04.422 INFO:teuthology.orchestra.run.smithi170.stdout:4451 write 0x5751200 thru 0x575a49c (0x929d bytes) 2024-09-21T23:17:04.423 INFO:teuthology.orchestra.run.smithi170.stdout:4453 read 0x58bd200 thru 0x58c552f (0x8330 bytes) 2024-09-21T23:17:04.423 INFO:teuthology.orchestra.run.smithi170.stdout:4456 read 0xeb6400 thru 0xeb64f9 (0xfa bytes) 2024-09-21T23:17:04.423 INFO:teuthology.orchestra.run.smithi170.stdout:4457 punch from 0x6aad600 to 0x6abca00, (0xf400 bytes) 2024-09-21T23:17:04.443 INFO:teuthology.orchestra.run.smithi170.stdout:4458 trunc from 0xe885e00 to 0x2830c00 2024-09-21T23:17:04.465 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:04.463+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:04.467 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:04.467 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:04.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:04.502 INFO:teuthology.orchestra.run.smithi170.stdout:4459 write 0xde31600 thru 0xde3ec77 (0xd678 bytes) 2024-09-21T23:17:04.502 INFO:teuthology.orchestra.run.smithi170.stdout:4463 read 0x3569000 thru 0x356996a (0x96b bytes) 2024-09-21T23:17:04.503 INFO:teuthology.orchestra.run.smithi170.stdout:4466 punch from 0xb716c00 to 0xb717e00, (0x1200 bytes) 2024-09-21T23:17:04.503 INFO:teuthology.orchestra.run.smithi170.stdout:4467 read 0x3a2200 thru 0x3a33d6 (0x11d7 bytes) 2024-09-21T23:17:04.504 INFO:teuthology.orchestra.run.smithi170.stdout:4469 read 0x65b0200 thru 0x65b9c71 (0x9a72 bytes) 2024-09-21T23:17:04.504 INFO:teuthology.orchestra.run.smithi170.stdout:4472 read 0x8f7b600 thru 0x8f7ce74 (0x1875 bytes) 2024-09-21T23:17:04.504 INFO:teuthology.orchestra.run.smithi170.stdout:4474 read 0x3de3e00 thru 0x3de8dc3 (0x4fc4 bytes) 2024-09-21T23:17:04.504 INFO:teuthology.orchestra.run.smithi170.stdout:4475 read 0xdb52800 thru 0xdb54c72 (0x2473 bytes) 2024-09-21T23:17:04.518 INFO:teuthology.orchestra.run.smithi170.stdout:4476 trunc from 0xde3ee00 to 0x4be5e00 2024-09-21T23:17:04.571 INFO:teuthology.orchestra.run.smithi170.stdout:4480 write 0xc51ee00 thru 0xc527f78 (0x9179 bytes) 2024-09-21T23:17:04.572 INFO:teuthology.orchestra.run.smithi170.stdout:4481 read 0x3e54400 thru 0x3e63ba0 (0xf7a1 bytes) 2024-09-21T23:17:04.572 INFO:teuthology.orchestra.run.smithi170.stdout:4484 punch from 0xc2dfe00 to 0xc2e4200, (0x4400 bytes) 2024-09-21T23:17:04.572 INFO:teuthology.orchestra.run.smithi170.stdout:4487 read 0xaf10c00 thru 0xaf18999 (0x7d9a bytes) 2024-09-21T23:17:04.574 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:04.572+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:04.592 INFO:teuthology.orchestra.run.smithi170.stdout:4489 write 0xc5af600 thru 0xc5b4412 (0x4e13 bytes) 2024-09-21T23:17:04.593 INFO:teuthology.orchestra.run.smithi170.stdout:4490 read 0xb81a000 thru 0xb81ddec (0x3ded bytes) 2024-09-21T23:17:04.593 INFO:teuthology.orchestra.run.smithi170.stdout:4491 punch from 0x1c33e00 to 0x1c35600, (0x1800 bytes) 2024-09-21T23:17:04.613 INFO:teuthology.orchestra.run.smithi170.stdout:4492 trunc from 0xc5b4600 to 0x70e600 2024-09-21T23:17:04.656 INFO:teuthology.orchestra.run.smithi170.stdout:4493 read 0x9e00 thru 0xe0b1 (0x42b2 bytes) 2024-09-21T23:17:04.666 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:04.663+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:04.670 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:04.670 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:04.670 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:04.670 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:04.670 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:04.686 INFO:teuthology.orchestra.run.smithi170.stdout:4494 write 0x8285400 thru 0x8289d15 (0x4916 bytes) 2024-09-21T23:17:04.686 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:04.687 INFO:teuthology.orchestra.run.smithi170.stdout:4496 read 0x6e0bc00 thru 0x6e1679f (0xaba0 bytes) 2024-09-21T23:17:04.701 INFO:teuthology.orchestra.run.smithi170.stdout:4500 trunc from 0x8289e00 to 0x15bbe00 2024-09-21T23:17:04.732 INFO:teuthology.orchestra.run.smithi170.stdout:4502 read 0xac0200 thru 0xacd470 (0xd271 bytes) 2024-09-21T23:17:04.733 INFO:teuthology.orchestra.run.smithi170.stdout:4503 punch from 0x1087e00 to 0x1096000, (0xe200 bytes) 2024-09-21T23:17:04.734 INFO:teuthology.orchestra.run.smithi170.stdout:4505 read 0x132c600 thru 0x13308ed (0x42ee bytes) 2024-09-21T23:17:04.734 INFO:teuthology.orchestra.run.smithi170.stdout:4506 read 0xd2ba00 thru 0xd2cce1 (0x12e2 bytes) 2024-09-21T23:17:04.734 INFO:teuthology.orchestra.run.smithi170.stdout:4507 write 0x141f800 thru 0x142086a (0x106b bytes) 2024-09-21T23:17:04.734 INFO:teuthology.orchestra.run.smithi170.stdout:4508 punch from 0x1165000 to 0x116ae00, (0x5e00 bytes) 2024-09-21T23:17:04.735 INFO:teuthology.orchestra.run.smithi170.stdout:4509 read 0x6f9400 thru 0x6fae72 (0x1a73 bytes) 2024-09-21T23:17:04.736 INFO:teuthology.orchestra.run.smithi170.stdout:4510 read 0x9d7e00 thru 0x9e6ad7 (0xecd8 bytes) 2024-09-21T23:17:04.736 INFO:teuthology.orchestra.run.smithi170.stdout:4512 punch from 0xfe2600 to 0xfed000, (0xaa00 bytes) 2024-09-21T23:17:04.737 INFO:teuthology.orchestra.run.smithi170.stdout:4513 read 0x2c1a00 thru 0x2c5764 (0x3d65 bytes) 2024-09-21T23:17:04.740 INFO:teuthology.orchestra.run.smithi170.stdout:4514 trunc from 0x15bbe00 to 0x7dd200 2024-09-21T23:17:04.773 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:04.772+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:04.780 INFO:teuthology.orchestra.run.smithi170.stdout:4515 punch from 0x56d400 to 0x578c00, (0xb800 bytes) 2024-09-21T23:17:04.800 INFO:teuthology.orchestra.run.smithi170.stdout:4516 trunc from 0x7dd200 to 0x744ac00 2024-09-21T23:17:04.821 INFO:teuthology.orchestra.run.smithi170.stdout:4517 trunc from 0x744ac00 to 0x89ff800 2024-09-21T23:17:04.838 INFO:teuthology.orchestra.run.smithi170.stdout:4518 write 0x43f4800 thru 0x43fa07a (0x587b bytes) 2024-09-21T23:17:04.840 INFO:teuthology.orchestra.run.smithi170.stdout:4522 read 0x7f6f800 thru 0x7f78ec6 (0x96c7 bytes) 2024-09-21T23:17:04.840 INFO:teuthology.orchestra.run.smithi170.stdout:4523 read 0x4336e00 thru 0x433e0f2 (0x72f3 bytes) 2024-09-21T23:17:04.840 INFO:teuthology.orchestra.run.smithi170.stdout:4524 read 0x58c8c00 thru 0x58cf61f (0x6a20 bytes) 2024-09-21T23:17:04.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:17:04.866 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:04.866 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:04.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:04.866 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:04.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.742865600000002, 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:17:04.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:17:04.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.872 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10124 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6156288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738507962, 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038330863999999999, 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10124 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125815+0000", 2024-09-21T23:17:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0399999999999995e-07 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125818+0000", 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6599999999999999e-06 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125827+0000", 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6330000000000002e-06 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125834+0000", 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9199999999999998e-06 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.163896+0000", 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038062072000000002 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164009+0000", 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011386400000000001 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164145+0000", 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013601100000000001 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10125 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6141952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738469592000001, 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038589275999999999, 2024-09-21T23:17:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10125 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125853+0000", 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125854+0000", 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125856+0000", 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8159999999999998e-06 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125865+0000", 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7739999999999993e-06 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125872+0000", 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164189+0000", 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038317276999999997 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164304+0000", 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114676 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164442+0000", 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013814 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10126 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6172672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738441809000001, 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038865093000000003, 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10126 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125881+0000", 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125884+0000", 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125893+0000", 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-21T23:17:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125900+0000", 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0500000000000003e-06 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164491+0000", 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038591473000000001 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164606+0000", 2024-09-21T23:17:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115109 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164746+0000", 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013963800000000001 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738414443000003, 2024-09-21T23:17:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:17:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738386970000001, 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:17:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738346796999998, 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738296028000001, 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:17:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:17:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738265445000003, 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:17:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738237538999996, 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738210318999997, 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738183399999997, 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:17:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738127636999998, 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:17:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738094464, 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:17:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738049244999999, 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.738009965000003, 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:17:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.737979479000003, 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.737950185999999, 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:17:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.737907843999999, 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:17:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:17:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.737524649000001, 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:17:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:17:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:17:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:17:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:04.959 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:04.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4525 write 0xb1b2800 thru 0xb1b3298 (0xa99 bytes) 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4526 trunc from 0xb1b3400 to 0xb578800 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4527 trunc from 0xb578800 to 0xc5b1a00 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4532 write 0xcc75200 thru 0xcc769c1 (0x17c2 bytes) 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4535 write 0x5813600 thru 0x581493f (0x1340 bytes) 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4536 read 0x896e00 thru 0x89e308 (0x7509 bytes) 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4537 read 0x8b37e00 thru 0x8b41bd4 (0x9dd5 bytes) 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4538 read 0x1130000 thru 0x113fd9a (0xfd9b bytes) 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4543 write 0x37d2400 thru 0x37dfa16 (0xd617 bytes) 2024-09-21T23:17:04.962 INFO:teuthology.orchestra.run.smithi170.stdout:4545 trunc from 0xcc76a00 to 0xb69a400 2024-09-21T23:17:04.963 INFO:teuthology.orchestra.run.smithi170.stdout:4550 read 0x2a59400 thru 0x2a66847 (0xd448 bytes) 2024-09-21T23:17:04.963 INFO:teuthology.orchestra.run.smithi170.stdout:4553 punch from 0x95b8e00 to 0x95c5a00, (0xcc00 bytes) 2024-09-21T23:17:04.963 INFO:teuthology.orchestra.run.smithi170.stdout:4554 read 0x8026000 thru 0x802c868 (0x6869 bytes) 2024-09-21T23:17:04.963 INFO:teuthology.orchestra.run.smithi170.stdout:4555 read 0x2122600 thru 0x2125307 (0x2d08 bytes) 2024-09-21T23:17:04.963 INFO:teuthology.orchestra.run.smithi170.stdout:4556 read 0x4d1c800 thru 0x4d25741 (0x8f42 bytes) 2024-09-21T23:17:04.963 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:04.959+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:04.964 INFO:teuthology.orchestra.run.smithi170.stdout:4557 trunc from 0xb69a400 to 0xddd0200 2024-09-21T23:17:04.978 INFO:teuthology.orchestra.run.smithi170.stdout:4561 write 0xb73a000 thru 0xb73fd47 (0x5d48 bytes) 2024-09-21T23:17:04.980 INFO:teuthology.orchestra.run.smithi170.stdout:4570 read 0x5579600 thru 0x557ffb5 (0x69b6 bytes) 2024-09-21T23:17:04.980 INFO:teuthology.orchestra.run.smithi170.stdout:4572 read 0xa94a200 thru 0xa954e2f (0xac30 bytes) 2024-09-21T23:17:04.980 INFO:teuthology.orchestra.run.smithi170.stdout:4574 read 0x9c19600 thru 0x9c24088 (0xaa89 bytes) 2024-09-21T23:17:04.983 INFO:teuthology.orchestra.run.smithi170.stdout:4575 trunc from 0xddd0200 to 0xc53dc00 2024-09-21T23:17:05.002 INFO:teuthology.orchestra.run.smithi170.stdout:4576 read 0x9088400 thru 0x9088cef (0x8f0 bytes) 2024-09-21T23:17:05.004 INFO:teuthology.orchestra.run.smithi170.stdout:4577 clone 27 order 21 su 131072 sc 11 2024-09-21T23:17:05.061 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:05.056+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:05.090 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:05.091 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:05.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:05.162 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:05.159+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:05.223 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:05.223 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:05.223 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:05.223 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:05.224 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:05.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:05.279 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:05.279+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:05.364 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:05.360+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:05.408 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 63.246555749999999, 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.181300991999997, 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.783958859999998, 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.780591170999998, 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.776481591, 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.772688670000001, 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.712529863, 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708772791999998, 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708745991000001, 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708719825999999, 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708691375999997, 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708600668000003, 2024-09-21T23:17:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708569089000001, 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708531399000002, 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708501009000003, 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708457592000002, 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.708422966999997, 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.705169466999997, 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.705134885, 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:05.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:05.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 47.489390485999998, 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:05.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:05.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:05.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:05.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:05.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:05.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.498 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:05.498 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:05.498 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:05.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:05.500 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:05.496+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:05.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:05.600 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:05.596+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:05.634 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:05.634 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:05.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:05.701 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:05.699+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:05.728 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:05.728 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:05.745 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:05.745 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:05.761 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:05.762 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:05.767 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:05.767 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:05.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:05.783 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:05.783 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:05.803 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:05.800+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:05.803 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:05.804 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:05.824 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:05.901 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:05.901 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:05.903 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:05.903+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:05.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:05.937 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:17:06.004 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:06.000+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:06.061 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:06.062 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:06.078 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:06.100 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone26 from 0xc53dc00 (overlap 0x70e600) to 0x0 2024-09-21T23:17:06.105 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:06.104+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:06.206 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:06.203+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:06.219 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:06.220 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:06.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:06.323 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:06.323+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:06.408 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:06.404+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:06.410 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:06.410 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:06.425 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:06.468 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:06.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:17:06.508 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:06.504+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:06.558 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:06.558 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:06.573 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:06.614 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:06.612+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:06.711 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:06.707+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:06.741 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:06.742 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:06.758 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:06.761 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-21T23:17:06.864 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:06.811+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:06.885 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:06.885 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:06.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:06.912 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:06.911+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:17:06.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:17:07.023 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:07.023+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:07.028 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:07.028 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:07.042 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:07.114 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:07.111+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:07.162 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:07.162 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:07.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:07.225 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:07.224+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:07.245 INFO:teuthology.orchestra.run.smithi170.stdout:4580 write 0xcafbe00 thru 0xcb03d4c (0x7f4d bytes) 2024-09-21T23:17:07.246 INFO:teuthology.orchestra.run.smithi170.stdout:4584 write 0x750d000 thru 0x750d9cf (0x9d0 bytes) 2024-09-21T23:17:07.246 INFO:teuthology.orchestra.run.smithi170.stdout:4585 read 0xc9f3a00 thru 0xc9f90b2 (0x56b3 bytes) 2024-09-21T23:17:07.246 INFO:teuthology.orchestra.run.smithi170.stdout:4587 write 0x8b75e00 thru 0x8b82ea1 (0xd0a2 bytes) 2024-09-21T23:17:07.246 INFO:teuthology.orchestra.run.smithi170.stdout:4590 punch from 0xc53bc00 to 0xc540c00, (0x5000 bytes) 2024-09-21T23:17:07.249 INFO:teuthology.orchestra.run.smithi170.stdout:4591 write 0x409be00 thru 0x40a3a77 (0x7c78 bytes) 2024-09-21T23:17:07.299 INFO:teuthology.orchestra.run.smithi170.stdout:4592 trunc from 0xcb03e00 to 0x470cc00 2024-09-21T23:17:07.306 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:07.306 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:07.315 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:07.315+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:07.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:07.390 INFO:teuthology.orchestra.run.smithi170.stdout:4593 write 0x74a6600 thru 0x74b46e9 (0xe0ea bytes) 2024-09-21T23:17:07.392 INFO:teuthology.orchestra.run.smithi170.stdout:4596 read 0x4417400 thru 0x441d51b (0x611c bytes) 2024-09-21T23:17:07.398 INFO:teuthology.orchestra.run.smithi170.stdout:4597 trunc from 0x74b4800 to 0x9d2d400 2024-09-21T23:17:07.416 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:07.415+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:07.418 INFO:teuthology.orchestra.run.smithi170.stdout:4598 write 0x80e1000 thru 0x80ebb70 (0xab71 bytes) 2024-09-21T23:17:07.419 INFO:teuthology.orchestra.run.smithi170.stdout:4599 read 0x8aab400 thru 0x8ab1bd7 (0x67d8 bytes) 2024-09-21T23:17:07.419 INFO:teuthology.orchestra.run.smithi170.stdout:4601 read 0x7c18200 thru 0x7c1f210 (0x7011 bytes) 2024-09-21T23:17:07.421 INFO:teuthology.orchestra.run.smithi170.stdout:4602 trunc from 0x9d2d400 to 0xa9c8200 2024-09-21T23:17:07.440 INFO:teuthology.orchestra.run.smithi170.stdout:4605 read 0x7a43800 thru 0x7a45350 (0x1b51 bytes) 2024-09-21T23:17:07.441 INFO:teuthology.orchestra.run.smithi170.stdout:4606 read 0x4ce8c00 thru 0x4ce95e2 (0x9e3 bytes) 2024-09-21T23:17:07.442 INFO:teuthology.orchestra.run.smithi170.stdout:4607 read 0x46df400 thru 0x46e1f94 (0x2b95 bytes) 2024-09-21T23:17:07.443 INFO:teuthology.orchestra.run.smithi170.stdout:4609 write 0x706a400 thru 0x7071eac (0x7aad bytes) 2024-09-21T23:17:07.444 INFO:teuthology.orchestra.run.smithi170.stdout:4610 read 0x69d2a00 thru 0x69dcb22 (0xa123 bytes) 2024-09-21T23:17:07.479 INFO:teuthology.orchestra.run.smithi170.stdout:4612 trunc from 0xa9c8200 to 0x81b600 2024-09-21T23:17:07.492 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:07.493 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:07.509 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:07.557 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:07.556+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:07.618 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:07.616+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:07.624 INFO:teuthology.orchestra.run.smithi170.stdout:4614 write 0x90b7a00 thru 0x90c52a1 (0xd8a2 bytes) 2024-09-21T23:17:07.633 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:07.633 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:07.639 INFO:teuthology.orchestra.run.smithi170.stdout:4616 trunc from 0x90c5400 to 0x35b5200 2024-09-21T23:17:07.647 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:07.684 INFO:teuthology.orchestra.run.smithi170.stdout:4617 read 0x2098800 thru 0x20a219d (0x999e bytes) 2024-09-21T23:17:07.687 INFO:teuthology.orchestra.run.smithi170.stdout:4620 read 0x1ee0200 thru 0x1ee6b80 (0x6981 bytes) 2024-09-21T23:17:07.719 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:07.719+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:07.731 INFO:teuthology.orchestra.run.smithi170.stdout:4622 write 0xbb61a00 thru 0xbb7122c (0xf82d bytes) 2024-09-21T23:17:07.740 INFO:teuthology.orchestra.run.smithi170.stdout:4624 trunc from 0xbb71400 to 0xd0cf400 2024-09-21T23:17:07.770 INFO:teuthology.orchestra.run.smithi170.stdout:4625 read 0xcfeb800 thru 0xcfecd2c (0x152d bytes) 2024-09-21T23:17:07.785 INFO:teuthology.orchestra.run.smithi170.stdout:4626 trunc from 0xd0cf400 to 0x6b19c00 2024-09-21T23:17:07.817 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:07.818 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:07.820 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:07.816+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:07.820 INFO:teuthology.orchestra.run.smithi170.stdout:4627 write 0x5f04600 thru 0x5f0aa0a (0x640b bytes) 2024-09-21T23:17:07.823 INFO:teuthology.orchestra.run.smithi170.stdout:4630 write 0x4595a00 thru 0x45a4876 (0xee77 bytes) 2024-09-21T23:17:07.823 INFO:teuthology.orchestra.run.smithi170.stdout:4633 punch from 0xfdfc00 to 0xfeac00, (0xb000 bytes) 2024-09-21T23:17:07.824 INFO:teuthology.orchestra.run.smithi170.stdout:4641 write 0x3561200 thru 0x35648b9 (0x36ba bytes) 2024-09-21T23:17:07.824 INFO:teuthology.orchestra.run.smithi170.stdout:4642 punch from 0x5bdd000 to 0x5be2000, (0x5000 bytes) 2024-09-21T23:17:07.824 INFO:teuthology.orchestra.run.smithi170.stdout:4644 punch from 0xe7c600 to 0xe81200, (0x4c00 bytes) 2024-09-21T23:17:07.825 INFO:teuthology.orchestra.run.smithi170.stdout:4645 write 0x4d68200 thru 0x4d6f2ee (0x70ef bytes) 2024-09-21T23:17:07.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:07.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:07.859 INFO:teuthology.orchestra.run.smithi170.stdout:4646 write 0x7a39a00 thru 0x7a3d57c (0x3b7d bytes) 2024-09-21T23:17:07.888 INFO:teuthology.orchestra.run.smithi170.stdout:4647 write 0x9d58c00 thru 0x9d63a89 (0xae8a bytes) 2024-09-21T23:17:07.909 INFO:teuthology.orchestra.run.smithi170.stdout:4649 trunc from 0x9d63c00 to 0x2be4200 2024-09-21T23:17:07.923 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:07.923+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:07.955 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:17:07.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:17:07.972 INFO:teuthology.orchestra.run.smithi170.stdout:4651 trunc from 0x2be4200 to 0xc4d0400 2024-09-21T23:17:07.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:17:07.985 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:07.986 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:07.993 INFO:teuthology.orchestra.run.smithi170.stdout:4652 read 0x7981200 thru 0x798c7b9 (0xb5ba bytes) 2024-09-21T23:17:07.996 INFO:teuthology.orchestra.run.smithi170.stdout:4653 punch from 0x709be00 to 0x70a8400, (0xc600 bytes) 2024-09-21T23:17:07.996 INFO:teuthology.orchestra.run.smithi170.stdout:4654 write 0xc076200 thru 0xc07f689 (0x948a bytes) 2024-09-21T23:17:07.996 INFO:teuthology.orchestra.run.smithi170.stdout:4657 write 0xc169600 thru 0xc16a2b9 (0xcba bytes) 2024-09-21T23:17:07.997 INFO:teuthology.orchestra.run.smithi170.stdout:4659 punch from 0x1daf600 to 0x1dbaa00, (0xb400 bytes) 2024-09-21T23:17:08.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:08.024 INFO:teuthology.orchestra.run.smithi170.stdout:4664 write 0xdeb0400 thru 0xdebbc87 (0xb888 bytes) 2024-09-21T23:17:08.025 INFO:teuthology.orchestra.run.smithi170.stdout:4667 write 0x5660000 thru 0x566c143 (0xc144 bytes) 2024-09-21T23:17:08.026 INFO:teuthology.orchestra.run.smithi170.stdout:4668 read 0x85b4c00 thru 0x85ba606 (0x5a07 bytes) 2024-09-21T23:17:08.026 INFO:teuthology.orchestra.run.smithi170.stdout:4669 read 0x9f1f600 thru 0x9f1f7f6 (0x1f7 bytes) 2024-09-21T23:17:08.026 INFO:teuthology.orchestra.run.smithi170.stdout:4670 write 0xd345200 thru 0xd353388 (0xe189 bytes) 2024-09-21T23:17:08.037 INFO:teuthology.orchestra.run.smithi170.stdout:4673 trunc from 0xdebbe00 to 0xa26e400 2024-09-21T23:17:08.045 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:08.045 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:08.045 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:08.043+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:08.067 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:08.067 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:08.089 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:08.089 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:08.105 INFO:teuthology.orchestra.run.smithi170.stdout:4674 write 0xdd39a00 thru 0xdd459ef (0xbff0 bytes) 2024-09-21T23:17:08.106 INFO:teuthology.orchestra.run.smithi170.stdout:4675 read 0x2bd1400 thru 0x2bdf3f7 (0xdff8 bytes) 2024-09-21T23:17:08.106 INFO:teuthology.orchestra.run.smithi170.stdout:4676 read 0x141c000 thru 0x1428fac (0xcfad bytes) 2024-09-21T23:17:08.107 INFO:teuthology.orchestra.run.smithi170.stdout:4680 read 0x1627800 thru 0x162ab47 (0x3348 bytes) 2024-09-21T23:17:08.108 INFO:teuthology.orchestra.run.smithi170.stdout:4682 read 0x48ff800 thru 0x4902755 (0x2f56 bytes) 2024-09-21T23:17:08.108 INFO:teuthology.orchestra.run.smithi170.stdout:4683 read 0xdd34000 thru 0xdd34212 (0x213 bytes) 2024-09-21T23:17:08.108 INFO:teuthology.orchestra.run.smithi170.stdout:4684 read 0x40bce00 thru 0x40c97ce (0xc9cf bytes) 2024-09-21T23:17:08.108 INFO:teuthology.orchestra.run.smithi170.stdout:4685 read 0x7bb4800 thru 0x7bc2ea8 (0xe6a9 bytes) 2024-09-21T23:17:08.109 INFO:teuthology.orchestra.run.smithi170.stdout:4686 write 0xbaaa800 thru 0xbab53d5 (0xabd6 bytes) 2024-09-21T23:17:08.109 INFO:teuthology.orchestra.run.smithi170.stdout:4687 read 0xa53a200 thru 0xa53fe89 (0x5c8a bytes) 2024-09-21T23:17:08.115 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:08.115 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:08.123 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:08.120+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:08.137 INFO:teuthology.orchestra.run.smithi170.stdout:4688 write 0xe44ba00 thru 0xe459034 (0xd635 bytes) 2024-09-21T23:17:08.139 INFO:teuthology.orchestra.run.smithi170.stdout:4692 punch from 0xce3ce00 to 0xce48c00, (0xbe00 bytes) 2024-09-21T23:17:08.139 INFO:teuthology.orchestra.run.smithi170.stdout:4693 read 0xcf4fa00 thru 0xcf5776e (0x7d6f bytes) 2024-09-21T23:17:08.139 INFO:teuthology.orchestra.run.smithi170.stdout:4694 read 0x3a26400 thru 0x3a279ce (0x15cf bytes) 2024-09-21T23:17:08.139 INFO:teuthology.orchestra.run.smithi170.stdout:4697 read 0x47b7e00 thru 0x47c5b9e (0xdd9f bytes) 2024-09-21T23:17:08.140 INFO:teuthology.orchestra.run.smithi170.stdout:4698 punch from 0x3bc3c00 to 0x3bca400, (0x6800 bytes) 2024-09-21T23:17:08.140 INFO:teuthology.orchestra.run.smithi170.stdout:4699 write 0xd7c9600 thru 0xd7d488b (0xb28c bytes) 2024-09-21T23:17:08.140 INFO:teuthology.orchestra.run.smithi170.stdout:4700 punch from 0x9d4fe00 to 0x9d55800, (0x5a00 bytes) 2024-09-21T23:17:08.140 INFO:teuthology.orchestra.run.smithi170.stdout:4701 read 0xc8df800 thru 0xc8ee8ad (0xf0ae bytes) 2024-09-21T23:17:08.141 INFO:teuthology.orchestra.run.smithi170.stdout:4702 punch from 0xc2a0e00 to 0xc2aa600, (0x9800 bytes) 2024-09-21T23:17:08.143 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:08.143 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:08.145 INFO:teuthology.orchestra.run.smithi170.stdout:4705 trunc from 0xe459200 to 0xc4f9e00 2024-09-21T23:17:08.179 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:08.179 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:08.184 INFO:teuthology.orchestra.run.smithi170.stdout:4706 read 0x860c200 thru 0x861b447 (0xf248 bytes) 2024-09-21T23:17:08.185 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:08.185 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:08.185 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:08.185 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:08.185 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:08.186 INFO:teuthology.orchestra.run.smithi170.stdout:4707 write 0x17aea00 thru 0x17b78c7 (0x8ec8 bytes) 2024-09-21T23:17:08.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:08.206 INFO:teuthology.orchestra.run.smithi170.stdout:4709 trunc from 0xc4f9e00 to 0x1799c00 2024-09-21T23:17:08.223 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:08.220+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:08.272 INFO:teuthology.orchestra.run.smithi170.stdout:4710 write 0x8d6be00 thru 0x8d6c291 (0x492 bytes) 2024-09-21T23:17:08.280 INFO:teuthology.orchestra.run.smithi170.stdout:4711 trunc from 0x8d6c400 to 0x73c6000 2024-09-21T23:17:08.322 INFO:teuthology.orchestra.run.smithi170.stdout:4712 trunc from 0x73c6000 to 0x192f200 2024-09-21T23:17:08.323 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:08.323+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:08.340 INFO:teuthology.orchestra.run.smithi170.stdout:4714 read 0x1650600 thru 0x165a6c7 (0xa0c8 bytes) 2024-09-21T23:17:08.360 INFO:teuthology.orchestra.run.smithi170.stdout:4715 trunc from 0x192f200 to 0xc8be400 2024-09-21T23:17:08.378 INFO:teuthology.orchestra.run.smithi170.stdout:4717 read 0xac18e00 thru 0xac1cf05 (0x4106 bytes) 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.224553494000006, 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.159298735999997, 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.761956603999998, 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.758588914999997, 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.754479334999999, 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.750686414, 2024-09-21T23:17:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.690527607, 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:17:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686770535999997, 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686743735, 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686717569999999, 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686689119999997, 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686598412000002, 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686566833000001, 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686529143000001, 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686498753000002, 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686455336000002, 2024-09-21T23:17:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.686420710999997, 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.683167210999997, 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.683132628999999, 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.467388229999997, 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.475 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:08.477 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:08.475+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:08.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:08.478 INFO:teuthology.orchestra.run.smithi170.stdout:4718 trunc from 0xc8be400 to 0x8c28000 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4722 read 0x7078800 thru 0x7086170 (0xd971 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4723 trunc from 0x8c28000 to 0x72a2200 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4725 read 0x3d1c200 thru 0x3d1dd10 (0x1b11 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4726 read 0x4a60a00 thru 0x4a63a11 (0x3012 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4727 read 0x5127800 thru 0x5128f48 (0x1749 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4730 write 0x4388e00 thru 0x438cd66 (0x3f67 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4732 trunc from 0x72a2200 to 0x8eb4c00 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4734 read 0x86ac00 thru 0x87a66f (0xfa70 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4735 read 0x3e8fc00 thru 0x3e9d14c (0xd54d bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4738 write 0x193cc00 thru 0x1942a58 (0x5e59 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4739 punch from 0x1e3c000 to 0x1e45e00, (0x9e00 bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4740 write 0x72bce00 thru 0x72bd16a (0x36b bytes) 2024-09-21T23:17:08.479 INFO:teuthology.orchestra.run.smithi170.stdout:4742 trunc from 0x8eb4c00 to 0x4691200 2024-09-21T23:17:08.504 INFO:teuthology.orchestra.run.smithi170.stdout:4745 read 0x2913000 thru 0x2920dd6 (0xddd7 bytes) 2024-09-21T23:17:08.538 INFO:teuthology.orchestra.run.smithi170.stdout:4746 write 0xa83e000 thru 0xa84a4b6 (0xc4b7 bytes) 2024-09-21T23:17:08.567 INFO:teuthology.orchestra.run.smithi170.stdout:4748 write 0xc879c00 thru 0xc8876a6 (0xdaa7 bytes) 2024-09-21T23:17:08.570 INFO:teuthology.orchestra.run.smithi170.stdout:4749 read 0xaa91200 thru 0xaa96064 (0x4e65 bytes) 2024-09-21T23:17:08.570 INFO:teuthology.orchestra.run.smithi170.stdout:4751 write 0x3b80e00 thru 0x3b82d4d (0x1f4e bytes) 2024-09-21T23:17:08.570 INFO:teuthology.orchestra.run.smithi170.stdout:4752 write 0x6e11a00 thru 0x6e1ac78 (0x9279 bytes) 2024-09-21T23:17:08.571 INFO:teuthology.orchestra.run.smithi170.stdout:4755 write 0x99ce00 thru 0x99d2f2 (0x4f3 bytes) 2024-09-21T23:17:08.573 INFO:teuthology.orchestra.run.smithi170.stdout:4758 write 0x2c33e00 thru 0x2c3601b (0x221c bytes) 2024-09-21T23:17:08.577 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:08.575+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:08.606 INFO:teuthology.orchestra.run.smithi170.stdout:4760 write 0xdf88000 thru 0xdf94da3 (0xcda4 bytes) 2024-09-21T23:17:08.608 INFO:teuthology.orchestra.run.smithi170.stdout:4762 read 0xbba6400 thru 0xbbb0bc8 (0xa7c9 bytes) 2024-09-21T23:17:08.608 INFO:teuthology.orchestra.run.smithi170.stdout:4764 read 0x56d8a00 thru 0x56d9535 (0xb36 bytes) 2024-09-21T23:17:08.609 INFO:teuthology.orchestra.run.smithi170.stdout:4766 punch from 0x5568800 to 0x5572800, (0xa000 bytes) 2024-09-21T23:17:08.610 INFO:teuthology.orchestra.run.smithi170.stdout:4768 read 0x3c02c00 thru 0x3c0c29c (0x969d bytes) 2024-09-21T23:17:08.610 INFO:teuthology.orchestra.run.smithi170.stdout:4769 read 0xce3da00 thru 0xce3fce0 (0x22e1 bytes) 2024-09-21T23:17:08.610 INFO:teuthology.orchestra.run.smithi170.stdout:4771 read 0x137da00 thru 0x1384696 (0x6c97 bytes) 2024-09-21T23:17:08.611 INFO:teuthology.orchestra.run.smithi170.stdout:4773 write 0x5a10c00 thru 0x5a20ae1 (0xfee2 bytes) 2024-09-21T23:17:08.611 INFO:teuthology.orchestra.run.smithi170.stdout:4774 read 0xab69400 thru 0xab6c1c8 (0x2dc9 bytes) 2024-09-21T23:17:08.637 INFO:teuthology.orchestra.run.smithi170.stdout:4776 trunc from 0xdf94e00 to 0x4317000 2024-09-21T23:17:08.665 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:08.665 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:08.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2444 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.660547+0000", 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0065384279999999998, 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006559385, 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2444 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660547+0000", 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660547+0000", 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660548+0000", 2024-09-21T23:17:08.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.094e-06 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660553+0000", 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9969999999999998e-06 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660565+0000", 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.219e-05 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660573+0000", 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18668 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.660814+0000", 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0062714299999999997, 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00634732, 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18668 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660814+0000", 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660814+0000", 2024-09-21T23:17:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660816+0000", 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.643e-06 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660824+0000", 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660842+0000", 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8485e-05 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660853+0000", 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0414999999999999e-05 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2445 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.661306+0000", 2024-09-21T23:17:08.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0057792620000000003, 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058998619999999996, 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2445 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661306+0000", 2024-09-21T23:17:08.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661306+0000", 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661307+0000", 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0589999999999999e-06 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661313+0000", 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4129999999999997e-06 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661326+0000", 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3013000000000001e-05 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661334+0000", 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9889999999999992e-06 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18669 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.661449+0000", 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0056361859999999996, 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005813403, 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18669 2024-09-21T23:17:08.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661449+0000", 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661449+0000", 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661451+0000", 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.212e-06 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661456+0000", 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6110000000000001e-06 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661471+0000", 2024-09-21T23:17:08.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4755e-05 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661478+0000", 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.357e-06 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18670 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.661820+0000", 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.005265741, 2024-09-21T23:17:08.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054870869999999999, 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18670 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661820+0000", 2024-09-21T23:17:08.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661820+0000", 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661821+0000", 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8700000000000004e-07 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661826+0000", 2024-09-21T23:17:08.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9019999999999998e-06 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661836+0000", 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0071000000000001e-05 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661842+0000", 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0190000000000003e-06 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2446 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.661860+0000", 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.005225904, 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0054924809999999996, 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2446 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.683 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661860+0000", 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661860+0000", 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661861+0000", 2024-09-21T23:17:08.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1969999999999999e-06 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661866+0000", 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1120000000000004e-06 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661879+0000", 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2621e-05 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.661887+0000", 2024-09-21T23:17:08.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2460000000000003e-06 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2447 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.662667+0000", 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0044183039999999996, 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047283159999999998, 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2447 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662667+0000", 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662667+0000", 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662668+0000", 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662673+0000", 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9049999999999996e-06 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662698+0000", 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5239999999999999e-05 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662705+0000", 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1189999999999999e-06 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18671 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.662803+0000", 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0042826119999999999, 2024-09-21T23:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046360489999999997, 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18671 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662803+0000", 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662803+0000", 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662804+0000", 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2200000000000003e-07 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662808+0000", 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.036e-06 2024-09-21T23:17:08.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662817+0000", 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8720000000000008e-06 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.662822+0000", 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.417e-06 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.692 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18672 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.663293+0000", 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0037927619999999999, 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041892350000000004, 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18672 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663293+0000", 2024-09-21T23:17:08.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663293+0000", 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663294+0000", 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5799999999999999e-06 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663301+0000", 2024-09-21T23:17:08.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663315+0000", 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3525e-05 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663324+0000", 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6579999999999997e-06 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2449 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.663284+0000", 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.003801291, 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042432019999999997, 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2449 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663284+0000", 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663284+0000", 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663286+0000", 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8199999999999999e-06 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663297+0000", 2024-09-21T23:17:08.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1199999999999999e-05 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663316+0000", 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8919999999999998e-05 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663327+0000", 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0591000000000001e-05 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2450 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.663628+0000", 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0034574010000000001, 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039424849999999999, 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2450 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.699 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663628+0000", 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663628+0000", 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663630+0000", 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-06 2024-09-21T23:17:08.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663637+0000", 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1289999999999998e-06 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663653+0000", 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5719999999999999e-05 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663662+0000", 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5570000000000001e-06 2024-09-21T23:17:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18673 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.663654+0000", 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0034313799999999999, 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039624789999999997, 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18673 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663654+0000", 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663654+0000", 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663656+0000", 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2929999999999999e-06 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663661+0000", 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.023e-06 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663672+0000", 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1277999999999999e-05 2024-09-21T23:17:08.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.663679+0000", 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2451 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.664272+0000", 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00281365, 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003387564, 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2451 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664272+0000", 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664272+0000", 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664273+0000", 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449999999999999e-06 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664279+0000", 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3199999999999996e-06 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664296+0000", 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6969999999999998e-05 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664309+0000", 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.26e-05 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18674 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.664277+0000", 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0028088399999999999, 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034257340000000002, 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18674 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664277+0000", 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664277+0000", 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664278+0000", 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6400000000000001e-07 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664282+0000", 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4469999999999999e-06 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664298+0000", 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5596000000000001e-05 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664309+0000", 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.169e-05 2024-09-21T23:17:08.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2452 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.664808+0000", 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0022779950000000001, 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029378590000000001, 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2452 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664808+0000", 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664808+0000", 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664808+0000", 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3099999999999996e-07 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664813+0000", 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5549999999999996e-06 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664824+0000", 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1104000000000001e-05 2024-09-21T23:17:08.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664830+0000", 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18675 3.1e 3.9df25cde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.664882+0000", 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0022035890000000002, 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002906343, 2024-09-21T23:17:08.715 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18675 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664882+0000", 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664882+0000", 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664883+0000", 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9899999999999999e-07 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664888+0000", 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9019999999999998e-06 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664899+0000", 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0752e-05 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.664905+0000", 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.718 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5414.0:2453 3.8 3:1038ece4:::rbd_data.10f192e7bdf.0000000000000022:head [truncate 131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.665796+0000", 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0012893259999999999, 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020435330000000002, 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5414", 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3355996929", 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2453 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665796+0000", 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665796+0000", 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665798+0000", 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5939999999999999e-06 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665806+0000", 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0539999999999995e-06 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665826+0000", 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0063000000000001e-05 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665837+0000", 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0805000000000001e-05 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665893+0000", 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5954e-05 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.665957+0000", 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3882999999999998e-05 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.666050+0000", 2024-09-21T23:17:08.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3633000000000003e-05 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.666860+0000", 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081003499999999997 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:18667 3.1e 3:7b3a4fb9:::rbd_header.10f192e7bdf:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:08.660409+0000", 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0066761570000000003, 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075007889999999999, 2024-09-21T23:17:08.723 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18667 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660409+0000", 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660409+0000", 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660411+0000", 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4759999999999999e-06 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660417+0000", 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6699999999999999e-06 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660428+0000", 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1552999999999999e-05 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.660435+0000", 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7220000000000002e-06 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.667575+0000", 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071403930000000001 2024-09-21T23:17:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:08.667766+0000", 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000190574 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 18 2024-09-21T23:17:08.727 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:08.728 INFO:teuthology.orchestra.run.smithi170.stdout:4777 read 0x180a00 thru 0x18dd22 (0xd323 bytes) 2024-09-21T23:17:08.728 INFO:teuthology.orchestra.run.smithi170.stdout:4781 trunc from 0x4317000 to 0xeda2200 2024-09-21T23:17:08.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:08.729 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:08.727+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:08.754 INFO:teuthology.orchestra.run.smithi170.stdout:4782 write 0x5721600 thru 0x57265dd (0x4fde bytes) 2024-09-21T23:17:08.755 INFO:teuthology.orchestra.run.smithi170.stdout:4783 read 0xaa9b600 thru 0xaa9dbc7 (0x25c8 bytes) 2024-09-21T23:17:08.756 INFO:teuthology.orchestra.run.smithi170.stdout:4788 write 0x37ab600 thru 0x37b8535 (0xcf36 bytes) 2024-09-21T23:17:08.756 INFO:teuthology.orchestra.run.smithi170.stdout:4789 write 0x3426000 thru 0x3429fb5 (0x3fb6 bytes) 2024-09-21T23:17:08.761 INFO:teuthology.orchestra.run.smithi170.stdout:4790 trunc from 0xeda2200 to 0xcaa0800 2024-09-21T23:17:08.817 INFO:teuthology.orchestra.run.smithi170.stdout:4792 write 0xdcbba00 thru 0xdcbd586 (0x1b87 bytes) 2024-09-21T23:17:08.817 INFO:teuthology.orchestra.run.smithi170.stdout:4794 read 0x2730e00 thru 0x2731862 (0xa63 bytes) 2024-09-21T23:17:08.817 INFO:teuthology.orchestra.run.smithi170.stdout:4795 read 0x91f6200 thru 0x91ffa8b (0x988c bytes) 2024-09-21T23:17:08.818 INFO:teuthology.orchestra.run.smithi170.stdout:4796 clone 28 order 21 su 131072 sc 2 2024-09-21T23:17:08.829 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:08.827+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:08.909 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:08.909 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:08.909 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:08.909 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:08.910 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:08.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:08.929 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:08.928+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:17:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:17:09.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:17:09.047 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:09.047+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:09.109 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:09.109 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:09.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349456476999997, 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.114 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.34941122, 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349361598000002, 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349331902999999, 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.124 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:09.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:09.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349298975000004, 2024-09-21T23:17:09.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:09.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349264679000001, 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:09.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:09.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.137 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349240172999998, 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.138 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:09.142 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349212182999999, 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:09.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:09.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349189154999998, 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349167645000001, 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349132103999999, 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:09.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:09.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.160 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349104466999997, 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:09.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:09.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349069303, 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:09.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.349034271999997, 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.169 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:09.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:09.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.348998887, 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.174 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:09.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:09.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.348958705000001, 2024-09-21T23:17:09.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:09.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:09.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.348921576999999, 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.348881501999998, 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.348856957999999, 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.348819173000003, 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.201 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:09.202 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone27 from 0xdcbd600 (overlap 0x81b600) to 0x74bc00 2024-09-21T23:17:09.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:09.203 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:09.199+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:09.303 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:09.303+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:09.372 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:09.372 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:09.372 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:09.373 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:09.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:09.404 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:09.400+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:09.505 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:09.503+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:09.563 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:09.564 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:09.564 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:09.564 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:09.564 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:09.578 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:09.619 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:09.619+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:09.707 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:09.704+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:09.747 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.825949179999995, 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824928333000003, 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824868660000007, 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824778683999995, 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824728773999993, 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824672633999995, 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824618850999997, 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824586574999998, 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824556294999994, 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824526638999998, 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824496671999995, 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824418843000004, 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824371135999996, 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824340939999999, 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824311023000007, 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:09.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:09.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824281223, 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:09.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:09.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824251290000007, 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:09.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:09.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:09.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824186327000007, 2024-09-21T23:17:09.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:09.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:09.843 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824155851, 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:09.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:09.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.824126687000003, 2024-09-21T23:17:09.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:09.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:09.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:09.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:09.853 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:09.854 INFO:teuthology.orchestra.run.smithi170.stdout:4797 write 0x8d63c00 thru 0x8d6515b (0x155c bytes) 2024-09-21T23:17:09.854 INFO:teuthology.orchestra.run.smithi170.stdout:4798 punch from 0x992a400 to 0x9935200, (0xae00 bytes) 2024-09-21T23:17:09.855 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:09.855 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:09.851+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:09.865 INFO:teuthology.orchestra.run.smithi170.stdout:4803 write 0xae1bc00 thru 0xae2157c (0x597d bytes) 2024-09-21T23:17:09.865 INFO:teuthology.orchestra.run.smithi170.stdout:4804 punch from 0xb73e400 to 0xb748200, (0x9e00 bytes) 2024-09-21T23:17:09.881 INFO:teuthology.orchestra.run.smithi170.stdout:4805 read 0x6694a00 thru 0x66a1167 (0xc768 bytes) 2024-09-21T23:17:09.909 INFO:teuthology.orchestra.run.smithi170.stdout:4808 write 0xed78200 thru 0xed79dfc (0x1bfd bytes) 2024-09-21T23:17:09.912 INFO:teuthology.orchestra.run.smithi170.stdout:4809 read 0x5cb5400 thru 0x5cba402 (0x5003 bytes) 2024-09-21T23:17:09.955 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:09.955+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:09.973 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:17:09.989 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:09.989 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:09.996 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:17:09.997 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:17:10.003 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:10.009 INFO:teuthology.orchestra.run.smithi170.stdout:4810 trunc from 0xed79e00 to 0x4864600 2024-09-21T23:17:10.056 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:10.052+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:10.066 INFO:teuthology.orchestra.run.smithi170.stdout:4811 trunc from 0x4864600 to 0xd256c00 2024-09-21T23:17:10.081 INFO:teuthology.orchestra.run.smithi170.stdout:4814 read 0x7f27e00 thru 0x7f2c3ff (0x4600 bytes) 2024-09-21T23:17:10.082 INFO:teuthology.orchestra.run.smithi170.stdout:4816 read 0xc283e00 thru 0xc28b113 (0x7314 bytes) 2024-09-21T23:17:10.082 INFO:teuthology.orchestra.run.smithi170.stdout:4817 punch from 0x3e50800 to 0x3e59a00, (0x9200 bytes) 2024-09-21T23:17:10.083 INFO:teuthology.orchestra.run.smithi170.stdout:4820 read 0x7009600 thru 0x700c964 (0x3365 bytes) 2024-09-21T23:17:10.083 INFO:teuthology.orchestra.run.smithi170.stdout:4821 clone 29 order 25 su 16777216 sc 14 2024-09-21T23:17:10.130 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:10.131 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:10.145 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:10.156 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:10.155+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:10.210 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:10.257 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:10.255+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:10.319 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:10.319 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:10.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:10.319 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:10.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 144.09641495100001, 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.58400966799999, 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.326 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.58394476500001, 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.331 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:10.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:10.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:10.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.58379832599999, 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:10.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:10.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:10.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.58372921099999, 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.342 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:10.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.583116886, 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:10.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.353 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.58291248099999, 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:10.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.582869695, 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.359 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:10.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:10.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.364 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19191140699999, 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.191824897, 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19174800799999, 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.376 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:10.381 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19154316800001, 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.382 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:10.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.191454886, 2024-09-21T23:17:10.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:10.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:10.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19132405799999, 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.393 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:10.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.398 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19127840499999, 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.399 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:10.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.191239978, 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:10.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19111051600001, 2024-09-21T23:17:10.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:10.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:10.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:10.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.191088874, 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.416 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:10.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:10.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:10.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19106883000001, 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:10.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 143.19098483799999, 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:10.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:10.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:10.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:10.433 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:10.434 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:10.435 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:10.436 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:17:10.437 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:17:10.437 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:10.438 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:10.438 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:10.439 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-21T23:17:09.973733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636697985,"num_objects":1353,"num_object_clones":369,"num_object_copies":2706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1353,"num_whiteouts":612,"num_read":256104,"num_read_kb":748563,"num_write":16124,"num_write_kb":98027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":23892960,"num_keys_recovered":282,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14515,"ondisk_log_size":14515,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":294396,"kb_used_data":79664,"kb_used_omap":10,"kb_used_meta":214645,"kb_avail":377192964,"statfs":{"total":386547056640,"available":386245595136,"internally_reserved":0,"allocated":81575936,"data_stored":92108482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":219797108},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18730769,"num_objects":99,"num_object_clones":42,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":51,"num_read":6444,"num_read_kb":24005,"num_write":743,"num_write_kb":5487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.034934"},"pg_stats":[{"pgid":"3.1f","version":"105'378","reported_seq":11824,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.966827+0000","last_change":"2024-09-21T23:16:36.051323+0000","last_active":"2024-09-21T23:17:05.966827+0000","last_peered":"2024-09-21T23:17:05.966827+0000","last_clean":"2024-09-21T23:17:05.966827+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:17:05.966827+0000","last_undegraded":"2024-09-21T23:17:05.966827+0000","last_fullsized":"2024-09-21T23:17:05.966827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:51:07.233469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":27626073,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":11540,"num_read_kb":18685,"num_write":490,"num_write_kb":2743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"109'374","reported_seq":4646,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.510471+0000","last_change":"2024-09-21T23:16:36.049621+0000","last_active":"2024-09-21T23:17:09.510471+0000","last_peered":"2024-09-21T23:17:09.510471+0000","last_clean":"2024-09-21T23:17:09.510471+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:17:09.510471+0000","last_undegraded":"2024-09-21T23:17:09.510471+0000","last_fullsized":"2024-09-21T23:17:09.510471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:08.653020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":23831107,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":4680,"num_read_kb":23197,"num_write":339,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"109'334","reported_seq":6427,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.876979+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:17:09.876979+0000","last_peered":"2024-09-21T23:17:09.876979+0000","last_clean":"2024-09-21T23:17:09.876979+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:17:09.876979+0000","last_undegraded":"2024-09-21T23:17:09.876979+0000","last_fullsized":"2024-09-21T23:17:09.876979+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:36:00.606066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":15382590,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":6645,"num_read_kb":20722,"num_write":244,"num_write_kb":2503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'595","reported_seq":9596,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.539649+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:17:09.539649+0000","last_peered":"2024-09-21T23:17:09.539649+0000","last_clean":"2024-09-21T23:17:09.539649+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:17:09.539649+0000","last_undegraded":"2024-09-21T23:17:09.539649+0000","last_fullsized":"2024-09-21T23:17:09.539649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:45.428957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":25558608,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":13129,"num_read_kb":35477,"num_write":431,"num_write_kb":11937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'310","reported_seq":3242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965452+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:17:05.965452+0000","last_peered":"2024-09-21T23:17:05.965452+0000","last_clean":"2024-09-21T23:17:05.965452+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:17:05.965452+0000","last_undegraded":"2024-09-21T23:17:05.965452+0000","last_fullsized":"2024-09-21T23:17:05.965452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:15:10.810891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":11233859,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":3766,"num_read_kb":14880,"num_write":319,"num_write_kb":1197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'388","reported_seq":8209,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.715147+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:17:09.715147+0000","last_peered":"2024-09-21T23:17:09.715147+0000","last_clean":"2024-09-21T23:17:09.715147+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:17:09.715147+0000","last_undegraded":"2024-09-21T23:17:09.715147+0000","last_fullsized":"2024-09-21T23:17:09.715147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:27:53.353847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":11005475,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":9340,"num_read_kb":23434,"num_write":294,"num_write_kb":1166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"109'271","reported_seq":2699,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.853326+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:17:09.853326+0000","last_peered":"2024-09-21T23:17:09.853326+0000","last_clean":"2024-09-21T23:17:09.853326+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:17:09.853326+0000","last_undegraded":"2024-09-21T23:17:09.853326+0000","last_fullsized":"2024-09-21T23:17:09.853326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:11.252794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":15388720,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":2291,"num_read_kb":10754,"num_write":223,"num_write_kb":1669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"106'497","reported_seq":17100,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.122155+0000","last_change":"2024-09-21T23:16:36.046053+0000","last_active":"2024-09-21T23:17:06.122155+0000","last_peered":"2024-09-21T23:17:06.122155+0000","last_clean":"2024-09-21T23:17:06.122155+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:17:06.122155+0000","last_undegraded":"2024-09-21T23:17:06.122155+0000","last_fullsized":"2024-09-21T23:17:06.122155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:15:51.084937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":44460663,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":18,"num_read":15663,"num_read_kb":44937,"num_write":626,"num_write_kb":9658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"106'275","reported_seq":12511,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.120820+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:17:06.120820+0000","last_peered":"2024-09-21T23:17:06.120820+0000","last_clean":"2024-09-21T23:17:06.120820+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:17:06.120820+0000","last_undegraded":"2024-09-21T23:17:06.120820+0000","last_fullsized":"2024-09-21T23:17:06.120820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:14:44.761782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":23820352,"num_objects":47,"num_object_clones":16,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":12084,"num_read_kb":30679,"num_write":240,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'442","reported_seq":5213,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985771+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:17:08.985771+0000","last_peered":"2024-09-21T23:17:08.985771+0000","last_clean":"2024-09-21T23:17:08.985771+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:17:08.985771+0000","last_undegraded":"2024-09-21T23:17:08.985771+0000","last_fullsized":"2024-09-21T23:17:08.985771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:36:46.141999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":45270178,"num_objects":78,"num_object_clones":29,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":41,"num_read":5178,"num_read_kb":21165,"num_write":500,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"106'263","reported_seq":3552,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.172674+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:17:06.172674+0000","last_peered":"2024-09-21T23:17:06.172674+0000","last_clean":"2024-09-21T23:17:06.172674+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:17:06.172674+0000","last_undegraded":"2024-09-21T23:17:06.172674+0000","last_fullsized":"2024-09-21T23:17:06.172674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:15.719902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287523,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":3577,"num_read_kb":26437,"num_write":313,"num_write_kb":3288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"107'609","reported_seq":4806,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000229+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:17:09.000229+0000","last_peered":"2024-09-21T23:17:09.000229+0000","last_clean":"2024-09-21T23:17:09.000229+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:17:09.000229+0000","last_undegraded":"2024-09-21T23:17:09.000229+0000","last_fullsized":"2024-09-21T23:17:09.000229+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:40:37.571093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14212255,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":6370,"num_read_kb":21180,"num_write":823,"num_write_kb":1942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000616+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:17:09.000616+0000","last_peered":"2024-09-21T23:17:09.000616+0000","last_clean":"2024-09-21T23:17:09.000616+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:17:09.000616+0000","last_undegraded":"2024-09-21T23:17:09.000616+0000","last_fullsized":"2024-09-21T23:17:09.000616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"106'641","reported_seq":19142,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965671+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:17:05.965671+0000","last_peered":"2024-09-21T23:17:05.965671+0000","last_clean":"2024-09-21T23:17:05.965671+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:17:05.965671+0000","last_undegraded":"2024-09-21T23:17:05.965671+0000","last_fullsized":"2024-09-21T23:17:05.965671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:41.497886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":30344787,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":20171,"num_read_kb":31198,"num_write":960,"num_write_kb":3898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.984030+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:17:08.984030+0000","last_peered":"2024-09-21T23:17:08.984030+0000","last_clean":"2024-09-21T23:17:08.984030+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:17:08.984030+0000","last_undegraded":"2024-09-21T23:17:08.984030+0000","last_fullsized":"2024-09-21T23:17:08.984030+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"108'1551","reported_seq":7206,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983660+0000","last_change":"2024-09-21T23:16:36.052892+0000","last_active":"2024-09-21T23:17:08.983660+0000","last_peered":"2024-09-21T23:17:08.983660+0000","last_clean":"2024-09-21T23:17:08.983660+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:17:08.983660+0000","last_undegraded":"2024-09-21T23:17:08.983660+0000","last_fullsized":"2024-09-21T23:17:08.983660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:19.426359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":11537834,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":12401,"num_read_kb":39577,"num_write":2717,"num_write_kb":6269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965571+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:17:05.965571+0000","last_peered":"2024-09-21T23:17:05.965571+0000","last_clean":"2024-09-21T23:17:05.965571+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:17:05.965571+0000","last_undegraded":"2024-09-21T23:17:05.965571+0000","last_fullsized":"2024-09-21T23:17:05.965571+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"109'332","reported_seq":1524,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983808+0000","last_change":"2024-09-21T23:16:36.050246+0000","last_active":"2024-09-21T23:17:08.983808+0000","last_peered":"2024-09-21T23:17:08.983808+0000","last_clean":"2024-09-21T23:17:08.983808+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:17:08.983808+0000","last_undegraded":"2024-09-21T23:17:08.983808+0000","last_fullsized":"2024-09-21T23:17:08.983808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:37:18.179842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":9044254,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":1919,"num_read_kb":3504,"num_write":365,"num_write_kb":1333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983832+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:17:08.983832+0000","last_peered":"2024-09-21T23:17:08.983832+0000","last_clean":"2024-09-21T23:17:08.983832+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:17:08.983832+0000","last_undegraded":"2024-09-21T23:17:08.983832+0000","last_fullsized":"2024-09-21T23:17:08.983832+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"109'428","reported_seq":8349,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.547558+0000","last_change":"2024-09-21T23:16:54.784043+0000","last_active":"2024-09-21T23:17:09.547558+0000","last_peered":"2024-09-21T23:17:09.547558+0000","last_clean":"2024-09-21T23:17:09.547558+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:17:09.547558+0000","last_undegraded":"2024-09-21T23:17:09.547558+0000","last_fullsized":"2024-09-21T23:17:09.547558+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:18:53.541977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":37682963,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":9293,"num_read_kb":58184,"num_write":515,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"106'510","reported_seq":5740,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000539+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:17:09.000539+0000","last_peered":"2024-09-21T23:17:09.000539+0000","last_clean":"2024-09-21T23:17:09.000539+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:17:09.000539+0000","last_undegraded":"2024-09-21T23:17:09.000539+0000","last_fullsized":"2024-09-21T23:17:09.000539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:38:39.354720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":13745729,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":19,"num_read":7418,"num_read_kb":21539,"num_write":504,"num_write_kb":1613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":222,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.984970+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:17:08.984970+0000","last_peered":"2024-09-21T23:17:08.984970+0000","last_clean":"2024-09-21T23:17:08.984970+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:17:08.984970+0000","last_undegraded":"2024-09-21T23:17:08.984970+0000","last_fullsized":"2024-09-21T23:17:08.984970+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"107'264","reported_seq":5413,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983379+0000","last_change":"2024-09-21T23:16:36.051625+0000","last_active":"2024-09-21T23:17:08.983379+0000","last_peered":"2024-09-21T23:17:08.983379+0000","last_clean":"2024-09-21T23:17:08.983379+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:17:08.983379+0000","last_undegraded":"2024-09-21T23:17:08.983379+0000","last_fullsized":"2024-09-21T23:17:08.983379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:23.583800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":14597748,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":5129,"num_read_kb":21281,"num_write":225,"num_write_kb":1017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983142+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:17:08.983142+0000","last_peered":"2024-09-21T23:17:08.983142+0000","last_clean":"2024-09-21T23:17:08.983142+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:17:08.983142+0000","last_undegraded":"2024-09-21T23:17:08.983142+0000","last_fullsized":"2024-09-21T23:17:08.983142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"106'311","reported_seq":3838,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.032535+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:17:06.032535+0000","last_peered":"2024-09-21T23:17:06.032535+0000","last_clean":"2024-09-21T23:17:06.032535+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:17:06.032535+0000","last_undegraded":"2024-09-21T23:17:06.032535+0000","last_fullsized":"2024-09-21T23:17:06.032535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:00:19.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":14083695,"num_objects":48,"num_object_clones":16,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":3432,"num_read_kb":13769,"num_write":302,"num_write_kb":4951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983131+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:17:08.983131+0000","last_peered":"2024-09-21T23:17:08.983131+0000","last_clean":"2024-09-21T23:17:08.983131+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:17:08.983131+0000","last_undegraded":"2024-09-21T23:17:08.983131+0000","last_fullsized":"2024-09-21T23:17:08.983131+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"109'421","reported_seq":12745,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983559+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:17:08.983559+0000","last_peered":"2024-09-21T23:17:08.983559+0000","last_clean":"2024-09-21T23:17:08.983559+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:17:08.983559+0000","last_undegraded":"2024-09-21T23:17:08.983559+0000","last_fullsized":"2024-09-21T23:17:08.983559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:58.771370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":22038192,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":13977,"num_read_kb":32770,"num_write":372,"num_write_kb":3526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":209,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.984816+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:17:08.984816+0000","last_peered":"2024-09-21T23:17:08.984816+0000","last_clean":"2024-09-21T23:17:08.984816+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:17:08.984816+0000","last_undegraded":"2024-09-21T23:17:08.984816+0000","last_fullsized":"2024-09-21T23:17:08.984816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":292,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965143+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:17:05.965143+0000","last_peered":"2024-09-21T23:17:05.965143+0000","last_clean":"2024-09-21T23:17:05.965143+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:17:05.965143+0000","last_undegraded":"2024-09-21T23:17:05.965143+0000","last_fullsized":"2024-09-21T23:17:05.965143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"109'783","reported_seq":11522,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.634089+0000","last_change":"2024-09-21T23:16:36.047835+0000","last_active":"2024-09-21T23:17:09.634089+0000","last_peered":"2024-09-21T23:17:09.634089+0000","last_clean":"2024-09-21T23:17:09.634089+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:17:09.634089+0000","last_undegraded":"2024-09-21T23:17:09.634089+0000","last_fullsized":"2024-09-21T23:17:09.634089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:49:29.936963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":31804221,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":12543,"num_read_kb":24107,"num_write":685,"num_write_kb":2366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"109'440","reported_seq":3974,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000818+0000","last_change":"2024-09-21T23:16:36.050207+0000","last_active":"2024-09-21T23:17:09.000818+0000","last_peered":"2024-09-21T23:17:09.000818+0000","last_clean":"2024-09-21T23:17:09.000818+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:17:09.000818+0000","last_undegraded":"2024-09-21T23:17:09.000818+0000","last_fullsized":"2024-09-21T23:17:09.000818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:52.198023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":15587543,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":22,"num_read":5217,"num_read_kb":11567,"num_write":356,"num_write_kb":1571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"109'229","reported_seq":2982,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.120995+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:17:09.120995+0000","last_peered":"2024-09-21T23:17:09.120995+0000","last_clean":"2024-09-21T23:17:09.120995+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:17:09.120995+0000","last_undegraded":"2024-09-21T23:17:09.120995+0000","last_fullsized":"2024-09-21T23:17:09.120995+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:23.004550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13881872,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":2485,"num_read_kb":14491,"num_write":212,"num_write_kb":1101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"109'492","reported_seq":7140,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.155338+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:17:09.155338+0000","last_peered":"2024-09-21T23:17:09.155338+0000","last_clean":"2024-09-21T23:17:09.155338+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:17:09.155338+0000","last_undegraded":"2024-09-21T23:17:09.155338+0000","last_fullsized":"2024-09-21T23:17:09.155338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:51:02.096060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":15633951,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":8982,"num_read_kb":18880,"num_write":347,"num_write_kb":1575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"108'378","reported_seq":3121,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985224+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:17:08.985224+0000","last_peered":"2024-09-21T23:17:08.985224+0000","last_clean":"2024-09-21T23:17:08.985224+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:17:08.985224+0000","last_undegraded":"2024-09-21T23:17:08.985224+0000","last_fullsized":"2024-09-21T23:17:08.985224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:54:44.861051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":18735661,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":3639,"num_read_kb":7414,"num_write":321,"num_write_kb":1951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"109'229","reported_seq":3805,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.873183+0000","last_change":"2024-09-21T23:16:36.052262+0000","last_active":"2024-09-21T23:17:09.873183+0000","last_peered":"2024-09-21T23:17:09.873183+0000","last_clean":"2024-09-21T23:17:09.873183+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:17:09.873183+0000","last_undegraded":"2024-09-21T23:17:09.873183+0000","last_fullsized":"2024-09-21T23:17:09.873183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:38.647370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9910540,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":3624,"num_read_kb":25002,"num_write":256,"num_write_kb":1283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"106'321","reported_seq":3208,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985803+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:17:08.985803+0000","last_peered":"2024-09-21T23:17:08.985803+0000","last_clean":"2024-09-21T23:17:08.985803+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:17:08.985803+0000","last_undegraded":"2024-09-21T23:17:08.985803+0000","last_fullsized":"2024-09-21T23:17:08.985803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:58:31.595771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":10085470,"num_objects":42,"num_object_clones":14,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":3877,"num_read_kb":11980,"num_write":315,"num_write_kb":1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"109'351","reported_seq":4626,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.703525+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:17:09.703525+0000","last_peered":"2024-09-21T23:17:09.703525+0000","last_clean":"2024-09-21T23:17:09.703525+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:17:09.703525+0000","last_undegraded":"2024-09-21T23:17:09.703525+0000","last_fullsized":"2024-09-21T23:17:09.703525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:44.790035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":14159940,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":6093,"num_read_kb":17934,"num_write":316,"num_write_kb":1053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"108'569","reported_seq":3599,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985477+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:17:08.985477+0000","last_peered":"2024-09-21T23:17:08.985477+0000","last_clean":"2024-09-21T23:17:08.985477+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:17:08.985477+0000","last_undegraded":"2024-09-21T23:17:08.985477+0000","last_fullsized":"2024-09-21T23:17:08.985477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:21.301225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":14623321,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":4297,"num_read_kb":16866,"num_write":779,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"109'434","reported_seq":4528,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.226941+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:17:09.226941+0000","last_peered":"2024-09-21T23:17:09.226941+0000","last_clean":"2024-09-21T23:17:09.226941+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:17:09.226941+0000","last_undegraded":"2024-09-21T23:17:09.226941+0000","last_fullsized":"2024-09-21T23:17:09.226941+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:32:14.673690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19277349,"num_objects":47,"num_object_clones":18,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":5023,"num_read_kb":22286,"num_write":583,"num_write_kb":2669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"106'618","reported_seq":20541,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.120302+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:17:06.120302+0000","last_peered":"2024-09-21T23:17:06.120302+0000","last_clean":"2024-09-21T23:17:06.120302+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:17:06.120302+0000","last_undegraded":"2024-09-21T23:17:06.120302+0000","last_fullsized":"2024-09-21T23:17:06.120302+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:42.768325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":28985959,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":23197,"num_read_kb":43001,"num_write":593,"num_write_kb":2732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'357","reported_seq":9196,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985072+0000","last_change":"2024-09-21T23:16:36.050076+0000","last_active":"2024-09-21T23:17:08.985072+0000","last_peered":"2024-09-21T23:17:08.985072+0000","last_clean":"2024-09-21T23:17:08.985072+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:17:08.985072+0000","last_undegraded":"2024-09-21T23:17:08.985072+0000","last_fullsized":"2024-09-21T23:17:08.985072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:05:56.608323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269166,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":9008,"num_read_kb":21566,"num_write":398,"num_write_kb":1946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":636107598,"num_objects":1349,"num_object_clones":369,"num_object_copies":2698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1349,"num_whiteouts":612,"num_read":255988,"num_read_kb":748463,"num_write":15963,"num_write_kb":95609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2034,"num_bytes_recovered":22712186,"num_keys_recovered":282,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":75911168,"data_stored":89374744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":14395,"ondisk_log_size":14395,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76584,"kb_used_data":23192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295256,"statfs":{"total":96636764160,"available":96558342144,"internally_reserved":0,"allocated":23748608,"data_stored":27806166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72424,"kb_used_data":19020,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299416,"statfs":{"total":96636764160,"available":96562601984,"internally_reserved":0,"allocated":19476480,"data_stored":22589763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607585,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607585,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84400,"kb_used_data":21280,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94287440,"statfs":{"total":96636764160,"available":96550338560,"internally_reserved":0,"allocated":21790720,"data_stored":24486092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005070,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60988,"kb_used_data":16172,"kb_used_omap":5,"kb_used_meta":44794,"kb_avail":94310852,"statfs":{"total":96636764160,"available":96574312448,"internally_reserved":0,"allocated":16560128,"data_stored":17226461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":45869349},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14729216,"data_stored":16127202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19988480,"data_stored":23433940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18255872,"data_stored":22112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22478848,"data_stored":27294232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:17:10.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:17:10.441 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:10.441 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:10.442 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:10.442 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:10.442 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:10.431+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:10.442 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:10.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:10.460 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:10.460 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:10.536 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:10.532+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:10.575 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:10.575 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:10.589 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:10.636 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:10.635+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:10.736 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:10.735+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:10.754 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:10.754 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:10.768 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:17:10.769 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:17:10.770 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:10.787 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-21T23:17:09.973733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636697985,"num_objects":1353,"num_object_clones":369,"num_object_copies":2706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1353,"num_whiteouts":612,"num_read":256104,"num_read_kb":748563,"num_write":16124,"num_write_kb":98027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":23892960,"num_keys_recovered":282,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14515,"ondisk_log_size":14515,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":294396,"kb_used_data":79664,"kb_used_omap":10,"kb_used_meta":214645,"kb_avail":377192964,"statfs":{"total":386547056640,"available":386245595136,"internally_reserved":0,"allocated":81575936,"data_stored":92108482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":219797108},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18730769,"num_objects":99,"num_object_clones":42,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":51,"num_read":6444,"num_read_kb":24005,"num_write":743,"num_write_kb":5487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.034934"},"pg_stats":[{"pgid":"3.1f","version":"105'378","reported_seq":11824,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.966827+0000","last_change":"2024-09-21T23:16:36.051323+0000","last_active":"2024-09-21T23:17:05.966827+0000","last_peered":"2024-09-21T23:17:05.966827+0000","last_clean":"2024-09-21T23:17:05.966827+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:17:05.966827+0000","last_undegraded":"2024-09-21T23:17:05.966827+0000","last_fullsized":"2024-09-21T23:17:05.966827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:51:07.233469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":27626073,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":11540,"num_read_kb":18685,"num_write":490,"num_write_kb":2743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"109'374","reported_seq":4646,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.510471+0000","last_change":"2024-09-21T23:16:36.049621+0000","last_active":"2024-09-21T23:17:09.510471+0000","last_peered":"2024-09-21T23:17:09.510471+0000","last_clean":"2024-09-21T23:17:09.510471+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:17:09.510471+0000","last_undegraded":"2024-09-21T23:17:09.510471+0000","last_fullsized":"2024-09-21T23:17:09.510471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:08.653020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":23831107,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":4680,"num_read_kb":23197,"num_write":339,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"109'334","reported_seq":6427,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.876979+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:17:09.876979+0000","last_peered":"2024-09-21T23:17:09.876979+0000","last_clean":"2024-09-21T23:17:09.876979+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:17:09.876979+0000","last_undegraded":"2024-09-21T23:17:09.876979+0000","last_fullsized":"2024-09-21T23:17:09.876979+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:36:00.606066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":15382590,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":6645,"num_read_kb":20722,"num_write":244,"num_write_kb":2503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'595","reported_seq":9596,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.539649+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:17:09.539649+0000","last_peered":"2024-09-21T23:17:09.539649+0000","last_clean":"2024-09-21T23:17:09.539649+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:17:09.539649+0000","last_undegraded":"2024-09-21T23:17:09.539649+0000","last_fullsized":"2024-09-21T23:17:09.539649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:45.428957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":25558608,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":13129,"num_read_kb":35477,"num_write":431,"num_write_kb":11937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'310","reported_seq":3242,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965452+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:17:05.965452+0000","last_peered":"2024-09-21T23:17:05.965452+0000","last_clean":"2024-09-21T23:17:05.965452+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:17:05.965452+0000","last_undegraded":"2024-09-21T23:17:05.965452+0000","last_fullsized":"2024-09-21T23:17:05.965452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:15:10.810891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":11233859,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":3766,"num_read_kb":14880,"num_write":319,"num_write_kb":1197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'388","reported_seq":8209,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.715147+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:17:09.715147+0000","last_peered":"2024-09-21T23:17:09.715147+0000","last_clean":"2024-09-21T23:17:09.715147+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:17:09.715147+0000","last_undegraded":"2024-09-21T23:17:09.715147+0000","last_fullsized":"2024-09-21T23:17:09.715147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:27:53.353847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":11005475,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":9340,"num_read_kb":23434,"num_write":294,"num_write_kb":1166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"109'271","reported_seq":2699,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.853326+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:17:09.853326+0000","last_peered":"2024-09-21T23:17:09.853326+0000","last_clean":"2024-09-21T23:17:09.853326+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:17:09.853326+0000","last_undegraded":"2024-09-21T23:17:09.853326+0000","last_fullsized":"2024-09-21T23:17:09.853326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:11.252794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":15388720,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":2291,"num_read_kb":10754,"num_write":223,"num_write_kb":1669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"106'497","reported_seq":17100,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.122155+0000","last_change":"2024-09-21T23:16:36.046053+0000","last_active":"2024-09-21T23:17:06.122155+0000","last_peered":"2024-09-21T23:17:06.122155+0000","last_clean":"2024-09-21T23:17:06.122155+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:17:06.122155+0000","last_undegraded":"2024-09-21T23:17:06.122155+0000","last_fullsized":"2024-09-21T23:17:06.122155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:15:51.084937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":44460663,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":18,"num_read":15663,"num_read_kb":44937,"num_write":626,"num_write_kb":9658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"106'275","reported_seq":12511,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.120820+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:17:06.120820+0000","last_peered":"2024-09-21T23:17:06.120820+0000","last_clean":"2024-09-21T23:17:06.120820+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:17:06.120820+0000","last_undegraded":"2024-09-21T23:17:06.120820+0000","last_fullsized":"2024-09-21T23:17:06.120820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:14:44.761782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":23820352,"num_objects":47,"num_object_clones":16,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":12084,"num_read_kb":30679,"num_write":240,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'442","reported_seq":5213,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985771+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:17:08.985771+0000","last_peered":"2024-09-21T23:17:08.985771+0000","last_clean":"2024-09-21T23:17:08.985771+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:17:08.985771+0000","last_undegraded":"2024-09-21T23:17:08.985771+0000","last_fullsized":"2024-09-21T23:17:08.985771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:36:46.141999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":45270178,"num_objects":78,"num_object_clones":29,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":41,"num_read":5178,"num_read_kb":21165,"num_write":500,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"106'263","reported_seq":3552,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.172674+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:17:06.172674+0000","last_peered":"2024-09-21T23:17:06.172674+0000","last_clean":"2024-09-21T23:17:06.172674+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:17:06.172674+0000","last_undegraded":"2024-09-21T23:17:06.172674+0000","last_fullsized":"2024-09-21T23:17:06.172674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:15.719902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287523,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":3577,"num_read_kb":26437,"num_write":313,"num_write_kb":3288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"107'609","reported_seq":4806,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000229+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:17:09.000229+0000","last_peered":"2024-09-21T23:17:09.000229+0000","last_clean":"2024-09-21T23:17:09.000229+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:17:09.000229+0000","last_undegraded":"2024-09-21T23:17:09.000229+0000","last_fullsized":"2024-09-21T23:17:09.000229+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:40:37.571093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14212255,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":6370,"num_read_kb":21180,"num_write":823,"num_write_kb":1942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000616+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:17:09.000616+0000","last_peered":"2024-09-21T23:17:09.000616+0000","last_clean":"2024-09-21T23:17:09.000616+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:17:09.000616+0000","last_undegraded":"2024-09-21T23:17:09.000616+0000","last_fullsized":"2024-09-21T23:17:09.000616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"106'641","reported_seq":19142,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965671+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:17:05.965671+0000","last_peered":"2024-09-21T23:17:05.965671+0000","last_clean":"2024-09-21T23:17:05.965671+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:17:05.965671+0000","last_undegraded":"2024-09-21T23:17:05.965671+0000","last_fullsized":"2024-09-21T23:17:05.965671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:41.497886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":30344787,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":20171,"num_read_kb":31198,"num_write":960,"num_write_kb":3898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.984030+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:17:08.984030+0000","last_peered":"2024-09-21T23:17:08.984030+0000","last_clean":"2024-09-21T23:17:08.984030+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:17:08.984030+0000","last_undegraded":"2024-09-21T23:17:08.984030+0000","last_fullsized":"2024-09-21T23:17:08.984030+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"108'1551","reported_seq":7206,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983660+0000","last_change":"2024-09-21T23:16:36.052892+0000","last_active":"2024-09-21T23:17:08.983660+0000","last_peered":"2024-09-21T23:17:08.983660+0000","last_clean":"2024-09-21T23:17:08.983660+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:17:08.983660+0000","last_undegraded":"2024-09-21T23:17:08.983660+0000","last_fullsized":"2024-09-21T23:17:08.983660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:19.426359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":11537834,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":12401,"num_read_kb":39577,"num_write":2717,"num_write_kb":6269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965571+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:17:05.965571+0000","last_peered":"2024-09-21T23:17:05.965571+0000","last_clean":"2024-09-21T23:17:05.965571+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:17:05.965571+0000","last_undegraded":"2024-09-21T23:17:05.965571+0000","last_fullsized":"2024-09-21T23:17:05.965571+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"109'332","reported_seq":1524,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983808+0000","last_change":"2024-09-21T23:16:36.050246+0000","last_active":"2024-09-21T23:17:08.983808+0000","last_peered":"2024-09-21T23:17:08.983808+0000","last_clean":"2024-09-21T23:17:08.983808+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:17:08.983808+0000","last_undegraded":"2024-09-21T23:17:08.983808+0000","last_fullsized":"2024-09-21T23:17:08.983808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:37:18.179842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":9044254,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":1919,"num_read_kb":3504,"num_write":365,"num_write_kb":1333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983832+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:17:08.983832+0000","last_peered":"2024-09-21T23:17:08.983832+0000","last_clean":"2024-09-21T23:17:08.983832+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:17:08.983832+0000","last_undegraded":"2024-09-21T23:17:08.983832+0000","last_fullsized":"2024-09-21T23:17:08.983832+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"109'428","reported_seq":8349,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.547558+0000","last_change":"2024-09-21T23:16:54.784043+0000","last_active":"2024-09-21T23:17:09.547558+0000","last_peered":"2024-09-21T23:17:09.547558+0000","last_clean":"2024-09-21T23:17:09.547558+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:17:09.547558+0000","last_undegraded":"2024-09-21T23:17:09.547558+0000","last_fullsized":"2024-09-21T23:17:09.547558+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:18:53.541977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":37682963,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":9293,"num_read_kb":58184,"num_write":515,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"106'510","reported_seq":5740,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000539+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:17:09.000539+0000","last_peered":"2024-09-21T23:17:09.000539+0000","last_clean":"2024-09-21T23:17:09.000539+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:17:09.000539+0000","last_undegraded":"2024-09-21T23:17:09.000539+0000","last_fullsized":"2024-09-21T23:17:09.000539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:38:39.354720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":13745729,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":19,"num_read":7418,"num_read_kb":21539,"num_write":504,"num_write_kb":1613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":222,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.984970+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:17:08.984970+0000","last_peered":"2024-09-21T23:17:08.984970+0000","last_clean":"2024-09-21T23:17:08.984970+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:17:08.984970+0000","last_undegraded":"2024-09-21T23:17:08.984970+0000","last_fullsized":"2024-09-21T23:17:08.984970+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"107'264","reported_seq":5413,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983379+0000","last_change":"2024-09-21T23:16:36.051625+0000","last_active":"2024-09-21T23:17:08.983379+0000","last_peered":"2024-09-21T23:17:08.983379+0000","last_clean":"2024-09-21T23:17:08.983379+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:17:08.983379+0000","last_undegraded":"2024-09-21T23:17:08.983379+0000","last_fullsized":"2024-09-21T23:17:08.983379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:23.583800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":14597748,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":5129,"num_read_kb":21281,"num_write":225,"num_write_kb":1017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983142+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:17:08.983142+0000","last_peered":"2024-09-21T23:17:08.983142+0000","last_clean":"2024-09-21T23:17:08.983142+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:17:08.983142+0000","last_undegraded":"2024-09-21T23:17:08.983142+0000","last_fullsized":"2024-09-21T23:17:08.983142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"106'311","reported_seq":3838,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.032535+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:17:06.032535+0000","last_peered":"2024-09-21T23:17:06.032535+0000","last_clean":"2024-09-21T23:17:06.032535+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:17:06.032535+0000","last_undegraded":"2024-09-21T23:17:06.032535+0000","last_fullsized":"2024-09-21T23:17:06.032535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:00:19.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":14083695,"num_objects":48,"num_object_clones":16,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":3432,"num_read_kb":13769,"num_write":302,"num_write_kb":4951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983131+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:17:08.983131+0000","last_peered":"2024-09-21T23:17:08.983131+0000","last_clean":"2024-09-21T23:17:08.983131+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:17:08.983131+0000","last_undegraded":"2024-09-21T23:17:08.983131+0000","last_fullsized":"2024-09-21T23:17:08.983131+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"109'421","reported_seq":12745,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.983559+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:17:08.983559+0000","last_peered":"2024-09-21T23:17:08.983559+0000","last_clean":"2024-09-21T23:17:08.983559+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:17:08.983559+0000","last_undegraded":"2024-09-21T23:17:08.983559+0000","last_fullsized":"2024-09-21T23:17:08.983559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:58.771370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":22038192,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":13977,"num_read_kb":32770,"num_write":372,"num_write_kb":3526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":209,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.984816+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:17:08.984816+0000","last_peered":"2024-09-21T23:17:08.984816+0000","last_clean":"2024-09-21T23:17:08.984816+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:17:08.984816+0000","last_undegraded":"2024-09-21T23:17:08.984816+0000","last_fullsized":"2024-09-21T23:17:08.984816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":292,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:05.965143+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:17:05.965143+0000","last_peered":"2024-09-21T23:17:05.965143+0000","last_clean":"2024-09-21T23:17:05.965143+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:17:05.965143+0000","last_undegraded":"2024-09-21T23:17:05.965143+0000","last_fullsized":"2024-09-21T23:17:05.965143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"109'783","reported_seq":11522,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.634089+0000","last_change":"2024-09-21T23:16:36.047835+0000","last_active":"2024-09-21T23:17:09.634089+0000","last_peered":"2024-09-21T23:17:09.634089+0000","last_clean":"2024-09-21T23:17:09.634089+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:17:09.634089+0000","last_undegraded":"2024-09-21T23:17:09.634089+0000","last_fullsized":"2024-09-21T23:17:09.634089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:49:29.936963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":31804221,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":12543,"num_read_kb":24107,"num_write":685,"num_write_kb":2366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"109'440","reported_seq":3974,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.000818+0000","last_change":"2024-09-21T23:16:36.050207+0000","last_active":"2024-09-21T23:17:09.000818+0000","last_peered":"2024-09-21T23:17:09.000818+0000","last_clean":"2024-09-21T23:17:09.000818+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:17:09.000818+0000","last_undegraded":"2024-09-21T23:17:09.000818+0000","last_fullsized":"2024-09-21T23:17:09.000818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:52.198023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":15587543,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":22,"num_read":5217,"num_read_kb":11567,"num_write":356,"num_write_kb":1571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"109'229","reported_seq":2982,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.120995+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:17:09.120995+0000","last_peered":"2024-09-21T23:17:09.120995+0000","last_clean":"2024-09-21T23:17:09.120995+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:17:09.120995+0000","last_undegraded":"2024-09-21T23:17:09.120995+0000","last_fullsized":"2024-09-21T23:17:09.120995+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:23.004550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13881872,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":2485,"num_read_kb":14491,"num_write":212,"num_write_kb":1101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"109'492","reported_seq":7140,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.155338+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:17:09.155338+0000","last_peered":"2024-09-21T23:17:09.155338+0000","last_clean":"2024-09-21T23:17:09.155338+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:17:09.155338+0000","last_undegraded":"2024-09-21T23:17:09.155338+0000","last_fullsized":"2024-09-21T23:17:09.155338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:51:02.096060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":15633951,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":8982,"num_read_kb":18880,"num_write":347,"num_write_kb":1575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"108'378","reported_seq":3121,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985224+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:17:08.985224+0000","last_peered":"2024-09-21T23:17:08.985224+0000","last_clean":"2024-09-21T23:17:08.985224+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:17:08.985224+0000","last_undegraded":"2024-09-21T23:17:08.985224+0000","last_fullsized":"2024-09-21T23:17:08.985224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:54:44.861051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":18735661,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":3639,"num_read_kb":7414,"num_write":321,"num_write_kb":1951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"109'229","reported_seq":3805,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.873183+0000","last_change":"2024-09-21T23:16:36.052262+0000","last_active":"2024-09-21T23:17:09.873183+0000","last_peered":"2024-09-21T23:17:09.873183+0000","last_clean":"2024-09-21T23:17:09.873183+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:17:09.873183+0000","last_undegraded":"2024-09-21T23:17:09.873183+0000","last_fullsized":"2024-09-21T23:17:09.873183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:05:38.647370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9910540,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":3624,"num_read_kb":25002,"num_write":256,"num_write_kb":1283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"106'321","reported_seq":3208,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985803+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:17:08.985803+0000","last_peered":"2024-09-21T23:17:08.985803+0000","last_clean":"2024-09-21T23:17:08.985803+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:17:08.985803+0000","last_undegraded":"2024-09-21T23:17:08.985803+0000","last_fullsized":"2024-09-21T23:17:08.985803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:58:31.595771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":10085470,"num_objects":42,"num_object_clones":14,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":3877,"num_read_kb":11980,"num_write":315,"num_write_kb":1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"109'351","reported_seq":4626,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.703525+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:17:09.703525+0000","last_peered":"2024-09-21T23:17:09.703525+0000","last_clean":"2024-09-21T23:17:09.703525+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:17:09.703525+0000","last_undegraded":"2024-09-21T23:17:09.703525+0000","last_fullsized":"2024-09-21T23:17:09.703525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:44.790035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":14159940,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":6093,"num_read_kb":17934,"num_write":316,"num_write_kb":1053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"108'569","reported_seq":3599,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985477+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:17:08.985477+0000","last_peered":"2024-09-21T23:17:08.985477+0000","last_clean":"2024-09-21T23:17:08.985477+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:17:08.985477+0000","last_undegraded":"2024-09-21T23:17:08.985477+0000","last_fullsized":"2024-09-21T23:17:08.985477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:21.301225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":14623321,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":4297,"num_read_kb":16866,"num_write":779,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"109'434","reported_seq":4528,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:09.226941+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:17:09.226941+0000","last_peered":"2024-09-21T23:17:09.226941+0000","last_clean":"2024-09-21T23:17:09.226941+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:17:09.226941+0000","last_undegraded":"2024-09-21T23:17:09.226941+0000","last_fullsized":"2024-09-21T23:17:09.226941+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:32:14.673690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19277349,"num_objects":47,"num_object_clones":18,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":5023,"num_read_kb":22286,"num_write":583,"num_write_kb":2669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"106'618","reported_seq":20541,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T23:17:06.120302+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:17:06.120302+0000","last_peered":"2024-09-21T23:17:06.120302+0000","last_clean":"2024-09-21T23:17:06.120302+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:17:06.120302+0000","last_undegraded":"2024-09-21T23:17:06.120302+0000","last_fullsized":"2024-09-21T23:17:06.120302+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:42.768325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":28985959,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":23197,"num_read_kb":43001,"num_write":593,"num_write_kb":2732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'357","reported_seq":9196,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T23:17:08.985072+0000","last_change":"2024-09-21T23:16:36.050076+0000","last_active":"2024-09-21T23:17:08.985072+0000","last_peered":"2024-09-21T23:17:08.985072+0000","last_clean":"2024-09-21T23:17:08.985072+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:17:08.985072+0000","last_undegraded":"2024-09-21T23:17:08.985072+0000","last_fullsized":"2024-09-21T23:17:08.985072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:05:56.608323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269166,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":9008,"num_read_kb":21566,"num_write":398,"num_write_kb":1946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":636107598,"num_objects":1349,"num_object_clones":369,"num_object_copies":2698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1349,"num_whiteouts":612,"num_read":255988,"num_read_kb":748463,"num_write":15963,"num_write_kb":95609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2034,"num_bytes_recovered":22712186,"num_keys_recovered":282,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":75911168,"data_stored":89374744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":14395,"ondisk_log_size":14395,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76584,"kb_used_data":23192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295256,"statfs":{"total":96636764160,"available":96558342144,"internally_reserved":0,"allocated":23748608,"data_stored":27806166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72424,"kb_used_data":19020,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299416,"statfs":{"total":96636764160,"available":96562601984,"internally_reserved":0,"allocated":19476480,"data_stored":22589763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607585,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607585,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84400,"kb_used_data":21280,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94287440,"statfs":{"total":96636764160,"available":96550338560,"internally_reserved":0,"allocated":21790720,"data_stored":24486092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005070,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60988,"kb_used_data":16172,"kb_used_omap":5,"kb_used_meta":44794,"kb_avail":94310852,"statfs":{"total":96636764160,"available":96574312448,"internally_reserved":0,"allocated":16560128,"data_stored":17226461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":45869349},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14729216,"data_stored":16127202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19988480,"data_stored":23433940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18255872,"data_stored":22112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22478848,"data_stored":27294232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:17:10.789 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:17:10.789 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:10.789 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:17:10.789 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:17:10.789 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:17:10.789 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:17:10.837 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:10.835+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:10.894 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:10.894 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:10.909 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:10.933 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:10.933 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:17:10.933 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:10.939 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:10.939+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:10.999 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:17:11.035 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:11.036 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:11.043 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:11.043+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:11.050 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:11.140 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:11.136+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:11.165 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone28 from 0xd256c00 (overlap 0x4864600) to 0xe7a600 2024-09-21T23:17:11.169 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:11.169 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:11.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:11.242 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:11.239+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:11.342 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:11.339+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:11.364 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:11.364 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:11.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:11.443 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:11.440+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:11.504 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:11.504 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:11.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:11.563 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:11.563+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:11.639 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:11.639 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:11.644 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:11.640+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:11.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:11.745 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:11.740+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:11.827 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:11.827 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:11.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:11.880 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-21T23:17:11.887 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:11.887+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:11.946 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:11.943+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:11.974 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:11.974 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:11.982 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:17:11.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:12.029 INFO:teuthology.orchestra.run.smithi170.stdout:4823 write 0x5764000 thru 0x5772eec (0xeeed bytes) 2024-09-21T23:17:12.034 INFO:teuthology.orchestra.run.smithi170.stdout:4824 write 0x8e3e00 thru 0x8e6668 (0x2869 bytes) 2024-09-21T23:17:12.041 INFO:teuthology.orchestra.run.smithi170.stdout:4825 read 0x2d69e00 thru 0x2d6e31f (0x4520 bytes) 2024-09-21T23:17:12.052 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:12.051+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:12.132 INFO:teuthology.orchestra.run.smithi170.stdout:4829 trunc from 0xd256c00 to 0x4093800 2024-09-21T23:17:12.147 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:12.144+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:12.186 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:12.186 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:12.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:12.251 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:12.251+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:12.331 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:12.331 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:12.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:12.350 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:12.348+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:12.364 INFO:teuthology.orchestra.run.smithi170.stdout:4830 read 0x114a200 thru 0x115453f (0xa340 bytes) 2024-09-21T23:17:12.386 INFO:teuthology.orchestra.run.smithi170.stdout:4831 write 0x529b800 thru 0x52ab0ca (0xf8cb bytes) 2024-09-21T23:17:12.432 INFO:teuthology.orchestra.run.smithi170.stdout:4832 write 0xccd5400 thru 0xccd7c01 (0x2802 bytes) 2024-09-21T23:17:12.450 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:12.447+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:12.459 INFO:teuthology.orchestra.run.smithi170.stdout:4833 trunc from 0xccd7e00 to 0xed5d400 2024-09-21T23:17:12.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:12.487 INFO:teuthology.orchestra.run.smithi170.stdout:4835 punch from 0xb64f000 to 0xb654e00, (0x5e00 bytes) 2024-09-21T23:17:12.489 INFO:teuthology.orchestra.run.smithi170.stdout:4837 clone 30 order 19 su 262144 sc 4 2024-09-21T23:17:12.539 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:12.539 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:12.551 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:12.547+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:12.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:12.631 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:12.631 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:12.650 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:12.650 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:12.651 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:12.647+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:12.666 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:12.667 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:12.688 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:12.688 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:12.708 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:12.709 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:12.721 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:12.721 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:12.721 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:12.721 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:12.721 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:12.729 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:12.730 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:12.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:12.792 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:12.791+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:12.852 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:12.848+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.995756286000002, 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:12.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:12.929 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994735438999996, 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994675766, 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:12.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:12.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.939 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994585790000002, 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.940 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994535880000001, 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:12.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:12.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994479740000003, 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:12.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994425957000004, 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:12.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:12.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994393681000005, 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:12.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994363401000001, 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:12.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994333745000006, 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:12.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:12.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994303778000003, 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.975 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:12.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:12.980 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994225948999997, 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:12.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:12.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994178242000004, 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:12.985 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:12.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:12.990 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994148046000007, 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:12.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:12.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:12.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994118129, 2024-09-21T23:17:12.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:12.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:12.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.000 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994088328999993, 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:13.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:13.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:13.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.994058396, 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:13.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:13.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:13.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.993993433, 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:13.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.993962956999994, 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:13.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:13.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:13.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:13.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.993933792999997, 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:13.021 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:13.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:13.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:13.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:13.026 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:13.027 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:13.028 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:13.024+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:13.129 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:13.127+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:13.163 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:13.164 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:13.164 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:13.179 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:13.215 INFO:teuthology.orchestra.run.smithi170.stdout:flattening image image_client.0-clone29 2024-09-21T23:17:13.229 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:13.224+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:13.318 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:13.318 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:13.319 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:13.319 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:13.319 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:13.330 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:13.328+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:13.341 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:13.431 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:13.428+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:13.503 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 147.28280782600001, 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.504 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:13.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.510 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.77040254299999, 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.511 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:13.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:13.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.77033764000001, 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:13.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:13.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:13.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.77019120099999, 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:13.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:13.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:13.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.77012208599999, 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.529 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:13.534 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.76950976099999, 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.535 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:13.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:13.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.540 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.76930535599999, 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.541 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:13.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.546 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.76926257, 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:13.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:13.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:13.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:13.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:13.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:13.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.37830428199999, 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:13.555 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:13.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.378217772, 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.561 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:13.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:13.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.37814088299999, 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:13.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.572 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.37793604300001, 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:13.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:13.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.377847761, 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:13.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:13.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:13.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:13.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.37771693299999, 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:13.584 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.37767127999999, 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.590 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:13.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:13.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.37763285299999, 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:13.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:13.601 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.377503391, 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:13.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:13.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.607 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.377481749, 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.608 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:13.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.613 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.377461705, 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.614 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:13.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:13.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 146.37737771299999, 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:13.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:13.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:13.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:13.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:13.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:13.625 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:13.626 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:13.627 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:13.628 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:13.623+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:13.727 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:13.724+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:13.798 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:13.798 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:13.798 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:13.798 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:13.812 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:13.828 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:13.827+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:13.928 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:13.927+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:13.977 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:13.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:13.977 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:13.978 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:13.978 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:13.992 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:14.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:17:14.037 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:14.036+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:14.123 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:14.123 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 99.166070411000007, 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:14.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:14.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:14.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:14.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 99.165803427, 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:14.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:14.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.553474882000003, 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:14.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.553432182999998, 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.145 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.553393563, 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:14.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.553361690999999, 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.150 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:14.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.553328467, 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.155 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:14.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:14.159 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.553293392, 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:14.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:14.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.553248433999997, 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:14.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.552796366999999, 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.169 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:14.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.552081888000004, 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.173 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:14.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.552040818000002, 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:14.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:14.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.551583700000002, 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.182 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.186 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.551347020999998, 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.187 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:14.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.550931439999999, 2024-09-21T23:17:14.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:14.195 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.550253779999998, 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:14.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:14.200 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.549762491000003, 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:14.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.204 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.549505592999999, 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:14.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.84031449400000002, 2024-09-21T23:17:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:17:14.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.83743579300000004, 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:17:14.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:17:14.219 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.220 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.220 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.220 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:14.221 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-21T23:17:14.222 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:14.222 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:14.216+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:14.322 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:14.319+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:14.354 INFO:teuthology.orchestra.run.smithi170.stdout:4839 read 0x2c8f800 thru 0x2c93cb2 (0x44b3 bytes) 2024-09-21T23:17:14.356 INFO:teuthology.orchestra.run.smithi170.stdout:4843 read 0x306e600 thru 0x307bcee (0xd6ef bytes) 2024-09-21T23:17:14.358 INFO:teuthology.orchestra.run.smithi170.stdout:4844 read 0xc0f3200 thru 0xc0fd087 (0x9e88 bytes) 2024-09-21T23:17:14.359 INFO:teuthology.orchestra.run.smithi170.stdout:4845 write 0x4e6aa00 thru 0x4e72f6c (0x856d bytes) 2024-09-21T23:17:14.362 INFO:teuthology.orchestra.run.smithi170.stdout:4846 read 0xbc12600 thru 0xbc2179d (0xf19e bytes) 2024-09-21T23:17:14.365 INFO:teuthology.orchestra.run.smithi170.stdout:4847 read 0x4f6da00 thru 0x4f78b14 (0xb115 bytes) 2024-09-21T23:17:14.366 INFO:teuthology.orchestra.run.smithi170.stdout:4848 punch from 0x4fe6800 to 0x4fea200, (0x3a00 bytes) 2024-09-21T23:17:14.390 INFO:teuthology.orchestra.run.smithi170.stdout:4849 read 0x574c00 thru 0x57f7b8 (0xabb9 bytes) 2024-09-21T23:17:14.399 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:14.399 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:14.399 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:14.399 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:14.413 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:14.414 INFO:teuthology.orchestra.run.smithi170.stdout:4852 trunc from 0xed5d400 to 0xd362800 2024-09-21T23:17:14.422 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:14.423+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:14.446 INFO:teuthology.orchestra.run.smithi170.stdout:4853 read 0x4bfce00 thru 0x4c0160c (0x480d bytes) 2024-09-21T23:17:14.449 INFO:teuthology.orchestra.run.smithi170.stdout:4854 read 0x5945200 thru 0x594ce61 (0x7c62 bytes) 2024-09-21T23:17:14.450 INFO:teuthology.orchestra.run.smithi170.stdout:4855 read 0x1977800 thru 0x1978932 (0x1133 bytes) 2024-09-21T23:17:14.523 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:14.523+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:14.530 INFO:teuthology.orchestra.run.smithi170.stdout:4856 trunc from 0xd362800 to 0x65bee00 2024-09-21T23:17:14.538 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:14.538 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:14.553 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:14.572 INFO:teuthology.orchestra.run.smithi170.stdout:4859 write 0x2fb2e00 thru 0x2fc0420 (0xd621 bytes) 2024-09-21T23:17:14.615 INFO:teuthology.orchestra.run.smithi170.stdout:4861 write 0xdf83600 thru 0xdf87bfb (0x45fc bytes) 2024-09-21T23:17:14.616 INFO:teuthology.orchestra.run.smithi170.stdout:4863 read 0x8687e00 thru 0x868def6 (0x60f7 bytes) 2024-09-21T23:17:14.616 INFO:teuthology.orchestra.run.smithi170.stdout:4864 punch from 0x849b200 to 0x84a2800, (0x7600 bytes) 2024-09-21T23:17:14.624 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:14.620+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:14.660 INFO:teuthology.orchestra.run.smithi170.stdout:4865 trunc from 0xdf87c00 to 0x10c8200 2024-09-21T23:17:14.691 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:17:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.563898081999994, 2024-09-21T23:17:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:17:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:17:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:17:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:17:14.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.698 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:14.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:14.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:17:14.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:14.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:17:14.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:17:14.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:17:14.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:14.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:17:14.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:17:14.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:14.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:17:14.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:17:14.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.710 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.710 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:14.710 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559446925000003, 2024-09-21T23:17:14.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:17:14.710 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.710 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.711 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.711 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:17:14.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.711 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:14.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:17:14.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:17:14.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:17:14.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:17:14.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:17:14.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:17:14.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:17:14.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:17:14.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:17:14.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:17:14.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:14.718 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559419452, 2024-09-21T23:17:14.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:17:14.719 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.719 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.719 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.719 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.719 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.719 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:14.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:17:14.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:17:14.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:17:14.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:17:14.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:17:14.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:17:14.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:17:14.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:17:14.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:17:14.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:17:14.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:17:14.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:17:14.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:17:14.726 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.726 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.726 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.727 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.727 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:14.727 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559379278999998, 2024-09-21T23:17:14.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:17:14.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.729 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:14.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:14.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:17:14.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:17:14.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:17:14.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:17:14.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:17:14.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:17:14.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:17:14.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:17:14.735 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.735 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.735 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.735 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.736 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:14.736 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.55932851, 2024-09-21T23:17:14.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:17:14.736 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.736 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.736 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.736 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.737 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:17:14.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.737 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:14.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:17:14.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:17:14.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:17:14.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:17:14.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:17:14.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:17:14.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:17:14.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:17:14.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.744 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.744 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:14.744 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559297927000003, 2024-09-21T23:17:14.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:17:14.744 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:17:14.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:17:14.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:17:14.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:17:14.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:17:14.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:17:14.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:17:14.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.750 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559270021000003, 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.751 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.752 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.752 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.752 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.752 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.752 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:17:14.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.752 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:17:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:17:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:17:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:17:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:17:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:17:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:17:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:17:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:17:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:17:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:17:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559242801000003, 2024-09-21T23:17:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:17:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:17:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559215881999997, 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:17:14.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:17:14.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559160118999998, 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:14.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:17:14.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.774 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559126945999999, 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:17:14.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:17:14.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559081726999999, 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.783 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559042447000003, 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:17:14.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:17:14.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.559011961000003, 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:14.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.558982667999999, 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:14.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:17:14.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.558940325999998, 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:17:14.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:17:14.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.558557131000001, 2024-09-21T23:17:14.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:14.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:17:14.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 1.433709458, 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:14.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:17:14.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.811 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 1.408660499, 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:17:14.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:17:14.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 1.3752273289999999, 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:14.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:14.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:14.822 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:14.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:14.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4867 punch from 0x115400 to 0x115a00, (0x600 bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4869 punch from 0xcc3a00 to 0xccd400, (0x9a00 bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4870 read 0xd30600 thru 0xd36a5a (0x645b bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4873 read 0xff7800 thru 0x1001466 (0x9c67 bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4874 read 0x746200 thru 0x752ae3 (0xc8e4 bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4876 punch from 0x474400 to 0x47d000, (0x8c00 bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4878 trunc from 0x10c8200 to 0xdf8b400 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4879 read 0x3d08000 thru 0x3d1061e (0x861f bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4880 read 0xb47a00 thru 0xb55629 (0xdc2a bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4882 read 0x81a3200 thru 0x81a4ec2 (0x1cc3 bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4884 read 0xb1e00 thru 0xbaccd (0x8ece bytes) 2024-09-21T23:17:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:4885 read 0x7e57e00 thru 0x7e5a406 (0x2607 bytes) 2024-09-21T23:17:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:4890 trunc from 0xdf8b400 to 0x8619000 2024-09-21T23:17:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:4891 punch from 0x3af3800 to 0x3af8600, (0x4e00 bytes) 2024-09-21T23:17:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:4892 write 0x1b33c00 thru 0x1b34141 (0x542 bytes) 2024-09-21T23:17:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:4893 write 0x4258400 thru 0x425cbd8 (0x47d9 bytes) 2024-09-21T23:17:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:4895 trunc from 0x8619000 to 0x7c4cc00 2024-09-21T23:17:14.827 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:14.823+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:14.853 INFO:teuthology.orchestra.run.smithi170.stdout:4896 read 0x1d80400 thru 0x1d8df64 (0xdb65 bytes) 2024-09-21T23:17:14.854 INFO:teuthology.orchestra.run.smithi170.stdout:4905 read 0x1068000 thru 0x107333f (0xb340 bytes) 2024-09-21T23:17:14.896 INFO:teuthology.orchestra.run.smithi170.stdout:4906 write 0xd9bce00 thru 0xd9c8e85 (0xc086 bytes) 2024-09-21T23:17:14.896 INFO:teuthology.orchestra.run.smithi170.stdout:4908 punch from 0x56c4200 to 0x56cda00, (0x9800 bytes) 2024-09-21T23:17:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:4909 read 0x3dd1400 thru 0x3dd3744 (0x2345 bytes) 2024-09-21T23:17:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:4912 read 0x8340c00 thru 0x8344b05 (0x3f06 bytes) 2024-09-21T23:17:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:4915 read 0x6d04800 thru 0x6d0a75b (0x5f5c bytes) 2024-09-21T23:17:14.908 INFO:teuthology.orchestra.run.smithi170.stdout:4916 trunc from 0xd9c9000 to 0x6cfc000 2024-09-21T23:17:14.924 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:14.923+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:14.968 INFO:teuthology.orchestra.run.smithi170.stdout:4917 write 0xbe89600 thru 0xbe94545 (0xaf46 bytes) 2024-09-21T23:17:14.973 INFO:teuthology.orchestra.run.smithi170.stdout:4918 trunc from 0xbe94600 to 0xa0fc000 2024-09-21T23:17:14.976 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:14.977 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:14.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5510.0:812 3.1f 3:fd03373d:::rbd_data.10f187bc4abf.000000000000017e:head [set-alloc-hint object_size 524288 write_size 524288,write 37888~45568 in=45568b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:14.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:14.979107+0000", 2024-09-21T23:17:14.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.000624469, 2024-09-21T23:17:14.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064653400000000004, 2024-09-21T23:17:14.977 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5510", 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/3700924744", 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 812 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979107+0000", 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979107+0000", 2024-09-21T23:17:14.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979109+0000", 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0320000000000002e-06 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979143+0000", 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3293000000000001e-05 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979155+0000", 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2884000000000001e-05 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979161+0000", 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979210+0000", 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8726000000000001e-05 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979309+0000", 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9178000000000005e-05 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:14.979392+0000", 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2890000000000001e-05 2024-09-21T23:17:14.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:14.984 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:14.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:14.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:14.984 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:17:14.984 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-21T23:17:14.984 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:14.988 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:14.988 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:14.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:15.006 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:15.006 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:4919 read 0x628f800 thru 0x62921b6 (0x29b7 bytes) 2024-09-21T23:17:15.008 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:17:15.015 INFO:teuthology.orchestra.run.smithi170.stdout:4920 write 0x3b1e800 thru 0x3b2200e (0x380f bytes) 2024-09-21T23:17:15.025 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:15.026 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:15.026 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:15.023+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:15.029 INFO:teuthology.orchestra.run.smithi170.stdout:4925 trunc from 0xa0fc000 to 0x2d70600 2024-09-21T23:17:15.053 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:15.053 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:15.069 INFO:teuthology.orchestra.run.smithi170.stdout:4926 read 0x1d1b000 thru 0x1d29447 (0xe448 bytes) 2024-09-21T23:17:15.070 INFO:teuthology.orchestra.run.smithi170.stdout:4927 read 0x2483200 thru 0x248d115 (0x9f16 bytes) 2024-09-21T23:17:15.070 INFO:teuthology.orchestra.run.smithi170.stdout:4928 punch from 0x27f9e00 to 0x2800800, (0x6a00 bytes) 2024-09-21T23:17:15.071 INFO:teuthology.orchestra.run.smithi170.stdout:4929 write 0x29d7400 thru 0x29ddaff (0x6700 bytes) 2024-09-21T23:17:15.071 INFO:teuthology.orchestra.run.smithi170.stdout:4933 read 0x2522800 thru 0x252ad8a (0x858b bytes) 2024-09-21T23:17:15.073 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:15.074 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:15.096 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:15.096 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:15.129 INFO:teuthology.orchestra.run.smithi170.stdout:4934 write 0xd74da00 thru 0xd75aedc (0xd4dd bytes) 2024-09-21T23:17:15.131 INFO:teuthology.orchestra.run.smithi170.stdout:4935 write 0x4f3f400 thru 0x4f40a1c (0x161d bytes) 2024-09-21T23:17:15.132 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:15.132 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:15.133 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:15.131+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:15.143 INFO:teuthology.orchestra.run.smithi170.stdout:4938 trunc from 0xd75b000 to 0x8b77e00 2024-09-21T23:17:15.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:15.200 INFO:teuthology.orchestra.run.smithi170.stdout:4939 trunc from 0x8b77e00 to 0x41bc600 2024-09-21T23:17:15.239 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:15.239+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:15.264 INFO:teuthology.orchestra.run.smithi170.stdout:4940 trunc from 0x41bc600 to 0xeccda00 2024-09-21T23:17:15.287 INFO:teuthology.orchestra.run.smithi170.stdout:4943 punch from 0x4d50200 to 0x4d51800, (0x1600 bytes) 2024-09-21T23:17:15.303 INFO:teuthology.orchestra.run.smithi170.stdout:4944 trunc from 0xeccda00 to 0x9661400 2024-09-21T23:17:15.318 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:15.318 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:15.327 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:15.323+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:15.332 INFO:teuthology.orchestra.run.smithi170.stdout:4948 read 0x261f000 thru 0x262b204 (0xc205 bytes) 2024-09-21T23:17:15.334 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:15.336 INFO:teuthology.orchestra.run.smithi170.stdout:4950 write 0xab6200 thru 0xab6241 (0x42 bytes) 2024-09-21T23:17:15.337 INFO:teuthology.orchestra.run.smithi170.stdout:4951 read 0x9f8a00 thru 0xa03bee (0xb1ef bytes) 2024-09-21T23:17:15.365 INFO:teuthology.orchestra.run.smithi170.stdout:4955 trunc from 0x9661400 to 0x33be00 2024-09-21T23:17:15.424 INFO:teuthology.orchestra.run.smithi170.stdout:4956 read 0x14400 thru 0x16ab6 (0x26b7 bytes) 2024-09-21T23:17:15.428 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:15.424+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:15.447 INFO:teuthology.orchestra.run.smithi170.stdout:4957 trunc from 0x33be00 to 0x7453000 2024-09-21T23:17:15.505 INFO:teuthology.orchestra.run.smithi170.stdout:4958 write 0xa7e0800 thru 0xa7eaa27 (0xa228 bytes) 2024-09-21T23:17:15.513 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:15.513 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:15.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:15.529 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:15.527+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:15.540 INFO:teuthology.orchestra.run.smithi170.stdout:4962 write 0xad50200 thru 0xad56d2f (0x6b30 bytes) 2024-09-21T23:17:15.541 INFO:teuthology.orchestra.run.smithi170.stdout:4965 read 0xa26f800 thru 0xa27c6dd (0xcede bytes) 2024-09-21T23:17:15.542 INFO:teuthology.orchestra.run.smithi170.stdout:4966 write 0x7ffc00 thru 0x80f065 (0xf466 bytes) 2024-09-21T23:17:15.542 INFO:teuthology.orchestra.run.smithi170.stdout:4967 punch from 0x9f70c00 to 0x9f7d200, (0xc600 bytes) 2024-09-21T23:17:15.543 INFO:teuthology.orchestra.run.smithi170.stdout:4968 read 0x33ccc00 thru 0x33d9d7f (0xd180 bytes) 2024-09-21T23:17:15.543 INFO:teuthology.orchestra.run.smithi170.stdout:4969 read 0x4f08200 thru 0x4f0df8b (0x5d8c bytes) 2024-09-21T23:17:15.543 INFO:teuthology.orchestra.run.smithi170.stdout:4970 punch from 0x785b600 to 0x785fe00, (0x4800 bytes) 2024-09-21T23:17:15.543 INFO:teuthology.orchestra.run.smithi170.stdout:4971 read 0x182b600 thru 0x1833fed (0x89ee bytes) 2024-09-21T23:17:15.545 INFO:teuthology.orchestra.run.smithi170.stdout:4973 trunc from 0xad56e00 to 0xb724800 2024-09-21T23:17:15.583 INFO:teuthology.orchestra.run.smithi170.stdout:4976 trunc from 0xb724800 to 0xeabdc00 2024-09-21T23:17:15.606 INFO:teuthology.orchestra.run.smithi170.stdout:4978 read 0xac3bc00 thru 0xac4125a (0x565b bytes) 2024-09-21T23:17:15.609 INFO:teuthology.orchestra.run.smithi170.stdout:4979 read 0x452f600 thru 0x4539d8b (0xa78c bytes) 2024-09-21T23:17:15.611 INFO:teuthology.orchestra.run.smithi170.stdout:4980 trunc from 0xeabdc00 to 0xdc70000 2024-09-21T23:17:15.629 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:15.624+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:15.638 INFO:teuthology.orchestra.run.smithi170.stdout:4981 write 0x7f6600 thru 0x7fae7e (0x487f bytes) 2024-09-21T23:17:15.641 INFO:teuthology.orchestra.run.smithi170.stdout:4982 read 0x80cb600 thru 0x80dac54 (0xf655 bytes) 2024-09-21T23:17:15.641 INFO:teuthology.orchestra.run.smithi170.stdout:4983 read 0xca98200 thru 0xcaa1281 (0x9082 bytes) 2024-09-21T23:17:15.641 INFO:teuthology.orchestra.run.smithi170.stdout:4985 write 0x2193400 thru 0x2195202 (0x1e03 bytes) 2024-09-21T23:17:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:4987 read 0x1bb8800 thru 0x1bbe1bd (0x59be bytes) 2024-09-21T23:17:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:4989 write 0x846cc00 thru 0x84771bb (0xa5bc bytes) 2024-09-21T23:17:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:4990 write 0xa923a00 thru 0xa925df4 (0x23f5 bytes) 2024-09-21T23:17:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:4991 punch from 0x3e20400 to 0x3e2e200, (0xde00 bytes) 2024-09-21T23:17:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:4992 read 0xc947c00 thru 0xc94b13b (0x353c bytes) 2024-09-21T23:17:15.648 INFO:teuthology.orchestra.run.smithi170.stdout:4993 trunc from 0xdc70000 to 0xa5a6a00 2024-09-21T23:17:15.692 INFO:teuthology.orchestra.run.smithi170.stdout:4999 punch from 0x3910200 to 0x3912400, (0x2200 bytes) 2024-09-21T23:17:15.696 INFO:teuthology.orchestra.run.smithi170.stdout:5000 read 0xc48400 thru 0xc50dba (0x89bb bytes) 2024-09-21T23:17:15.697 INFO:teuthology.orchestra.run.smithi170.stdout:5001 write 0x33a5c00 thru 0x33b528a (0xf68b bytes) 2024-09-21T23:17:15.697 INFO:teuthology.orchestra.run.smithi170.stdout:5003 read 0x6498c00 thru 0x64a4ef2 (0xc2f3 bytes) 2024-09-21T23:17:15.697 INFO:teuthology.orchestra.run.smithi170.stdout:5007 read 0x94a200 thru 0x9556dd (0xb4de bytes) 2024-09-21T23:17:15.700 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:15.701 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:15.717 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:15.730 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:15.728+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:15.737 INFO:teuthology.orchestra.run.smithi170.stdout:5008 write 0xd326400 thru 0xd327bd3 (0x17d4 bytes) 2024-09-21T23:17:15.747 INFO:teuthology.orchestra.run.smithi170.stdout:5009 trunc from 0xd327c00 to 0xa7c0800 2024-09-21T23:17:15.784 INFO:teuthology.orchestra.run.smithi170.stdout:5011 write 0x1d61200 thru 0x1d639af (0x27b0 bytes) 2024-09-21T23:17:15.793 INFO:teuthology.orchestra.run.smithi170.stdout:5014 trunc from 0xa7c0800 to 0x8c68000 2024-09-21T23:17:15.831 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:15.828+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:15.834 INFO:teuthology.orchestra.run.smithi170.stdout:5019 write 0x6eb3200 thru 0x6eb708e (0x3e8f bytes) 2024-09-21T23:17:15.836 INFO:teuthology.orchestra.run.smithi170.stdout:5021 write 0x5fc6200 thru 0x5fcc6ee (0x64ef bytes) 2024-09-21T23:17:15.837 INFO:teuthology.orchestra.run.smithi170.stdout:5022 write 0x1ba0000 thru 0x1ba0800 (0x801 bytes) 2024-09-21T23:17:15.851 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:15.851 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:15.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:15.881 INFO:teuthology.orchestra.run.smithi170.stdout:5023 write 0xc225000 thru 0xc22d727 (0x8728 bytes) 2024-09-21T23:17:15.914 INFO:teuthology.orchestra.run.smithi170.stdout:5024 write 0xdb80000 thru 0xdb80c41 (0xc42 bytes) 2024-09-21T23:17:15.916 INFO:teuthology.orchestra.run.smithi170.stdout:5026 write 0xad8fa00 thru 0xad9e170 (0xe771 bytes) 2024-09-21T23:17:15.916 INFO:teuthology.orchestra.run.smithi170.stdout:5027 read 0xbfd0600 thru 0xbfd0b14 (0x515 bytes) 2024-09-21T23:17:15.924 INFO:teuthology.orchestra.run.smithi170.stdout:5031 trunc from 0xdb80e00 to 0x9f91000 2024-09-21T23:17:15.932 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:15.928+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:15.948 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:15.949 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:15.949 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T23:17:15.949 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:17:15.949 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:17:15.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T23:17:15.958 INFO:teuthology.orchestra.run.smithi170.stdout:5033 read 0x8e86400 thru 0x8e8a02e (0x3c2f bytes) 2024-09-21T23:17:15.961 INFO:teuthology.orchestra.run.smithi170.stdout:5035 write 0x9a74000 thru 0x9a77bd0 (0x3bd1 bytes) 2024-09-21T23:17:15.961 INFO:teuthology.orchestra.run.smithi170.stdout:5036 read 0x813e800 thru 0x8144494 (0x5c95 bytes) 2024-09-21T23:17:15.962 INFO:teuthology.orchestra.run.smithi170.stdout:5039 write 0x76cae00 thru 0x76d9704 (0xe905 bytes) 2024-09-21T23:17:15.962 INFO:teuthology.orchestra.run.smithi170.stdout:5040 write 0x3e1fc00 thru 0x3e2e437 (0xe838 bytes) 2024-09-21T23:17:16.002 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:16.002 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:16.004 INFO:teuthology.orchestra.run.smithi170.stdout:5044 write 0xd9f7000 thru 0xd9fa8dc (0x38dd bytes) 2024-09-21T23:17:16.005 INFO:teuthology.orchestra.run.smithi170.stdout:5046 punch from 0x7abf800 to 0x7ac8c00, (0x9400 bytes) 2024-09-21T23:17:16.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:16.023 INFO:teuthology.orchestra.run.smithi170.stdout:5047 trunc from 0xd9faa00 to 0x7131e00 2024-09-21T23:17:16.034 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:16.032+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:16.065 INFO:teuthology.orchestra.run.smithi170.stdout:5048 read 0x2fd4400 thru 0x2fe3e85 (0xfa86 bytes) 2024-09-21T23:17:16.067 INFO:teuthology.orchestra.run.smithi170.stdout:5052 read 0x6e84a00 thru 0x6e84a58 (0x59 bytes) 2024-09-21T23:17:16.077 INFO:teuthology.orchestra.run.smithi170.stdout:5055 trunc from 0x7131e00 to 0xa7c6200 2024-09-21T23:17:16.080 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:16.080 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "" 2024-09-21T23:17:16.080 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:16.102 INFO:teuthology.orchestra.run.smithi170.stdout:5056 read 0x6b35e00 thru 0x6b41b20 (0xbd21 bytes) 2024-09-21T23:17:16.104 INFO:teuthology.orchestra.run.smithi170.stdout:5059 punch from 0x474be00 to 0x474cc00, (0xe00 bytes) 2024-09-21T23:17:16.134 INFO:teuthology.orchestra.run.smithi170.stdout:5062 write 0xc011a00 thru 0xc017f0e (0x650f bytes) 2024-09-21T23:17:16.134 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:16.132+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:16.135 INFO:teuthology.orchestra.run.smithi170.stdout:5063 punch from 0x37ca00 to 0x37f200, (0x2800 bytes) 2024-09-21T23:17:16.136 INFO:teuthology.orchestra.run.smithi170.stdout:5066 read 0x7d8c400 thru 0x7d96e6d (0xaa6e bytes) 2024-09-21T23:17:16.136 INFO:teuthology.orchestra.run.smithi170.stdout:5067 write 0x9d96400 thru 0x9d9df66 (0x7b67 bytes) 2024-09-21T23:17:16.136 INFO:teuthology.orchestra.run.smithi170.stdout:5070 write 0x1be8400 thru 0x1becdd8 (0x49d9 bytes) 2024-09-21T23:17:16.142 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:16.142 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:5071 trunc from 0xc018000 to 0x17e1c00 2024-09-21T23:17:16.157 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:16.194 INFO:teuthology.orchestra.run.smithi170.stdout:5072 read 0xed8200 thru 0xee2bbd (0xa9be bytes) 2024-09-21T23:17:16.199 INFO:teuthology.orchestra.run.smithi170.stdout:5073 read 0x1250a00 thru 0x125f9bb (0xefbc bytes) 2024-09-21T23:17:16.199 INFO:teuthology.orchestra.run.smithi170.stdout:5074 punch from 0xc9200 to 0xd9000, (0xfe00 bytes) 2024-09-21T23:17:16.200 INFO:teuthology.orchestra.run.smithi170.stdout:5076 read 0xfaa200 thru 0xfb177e (0x757f bytes) 2024-09-21T23:17:16.200 INFO:teuthology.orchestra.run.smithi170.stdout:5083 read 0xa59c00 thru 0xa603d5 (0x67d6 bytes) 2024-09-21T23:17:16.200 INFO:teuthology.orchestra.run.smithi170.stdout:5084 read 0xb1c200 thru 0xb2553c (0x933d bytes) 2024-09-21T23:17:16.215 INFO:teuthology.orchestra.run.smithi170.stdout:5085 trunc from 0x17e1c00 to 0x9309000 2024-09-21T23:17:16.235 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:16.231+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:16.242 INFO:teuthology.orchestra.run.smithi170.stdout:5086 write 0x2570600 thru 0x257a756 (0xa157 bytes) 2024-09-21T23:17:16.243 INFO:teuthology.orchestra.run.smithi170.stdout:5088 punch from 0x3d4600 to 0x3e3400, (0xee00 bytes) 2024-09-21T23:17:16.244 INFO:teuthology.orchestra.run.smithi170.stdout:5092 punch from 0x7549800 to 0x7556000, (0xc800 bytes) 2024-09-21T23:17:16.244 INFO:teuthology.orchestra.run.smithi170.stdout:5093 clone 31 order 24 su 131072 sc 6 2024-09-21T23:17:16.287 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:16.288 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:16.302 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:16.343 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:16.343+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:16.434 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:16.434 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:16.435 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:16.435+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:16.448 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:16.536 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:16.539+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:16.570 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:16.570 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:16.584 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:16.637 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:16.632+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:16.700 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:16.701 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:16.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:16.757 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:16.756+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:16.839 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:16.836+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:16.878 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:16.878 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:16.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:16.940 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:16.939+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:17.032 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:17:17.041 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:17.036+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:17.063 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:17.063 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:17.078 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:17.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:17.156 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone30 from 0x9309000 (overlap 0x33be00) to 0x0 2024-09-21T23:17:17.167 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:17.164+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:17.242 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:17.239+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:17.251 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:17.252 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:17.266 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:17.270 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:17.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:17.287 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:17.287 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:17.303 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:17.304 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:17.323 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:17.323 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:17.343 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:17.343+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:17.344 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:17.344 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:17.364 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:17.364 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:17.428 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:17.428 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:17.428 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:17.428 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:17.442 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:17.444 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:17.440+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:17.544 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:17.540+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:17.568 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:17.568 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:17.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:17.670 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:17.668+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:17.684 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-21T23:17:17.746 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:17.743+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.628165671000005, 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.753 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.758 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623714514, 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623687040999997, 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.763 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:17:17.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:17:17.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623646867999994, 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623596098999997, 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623565515999999, 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:17:17.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:17:17.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.62353761, 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:17:17.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:17:17.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:17:17.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:17:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623510390000007, 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:17:17.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:17:17.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:17:17.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:17:17.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623483471, 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.787 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:17:17.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:17:17.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623427707999994, 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623394535000003, 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.795 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:17.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:17:17.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:17:17.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623349316000002, 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.799 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623310036000007, 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:17:17.803 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623279550000007, 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:17:17.807 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623250256999995, 2024-09-21T23:17:17.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:17:17.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.623207914999995, 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:17:17.815 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:17.819 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 64.622824719999997, 2024-09-21T23:17:17.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:17:17.820 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.821 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:17:17.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:17:17.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.824 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.497977047, 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:17.825 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:17:17.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:17:17.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:17:17.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.4729280879999997, 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:17:17.830 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:17:17.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:17:17.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:17:17.835 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.4394949180000003, 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:17.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:17.839 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:17.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:17.882 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:17.880+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:17.948 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:17.947+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:17.977 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:17.977 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:17.977 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:17.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:18.019 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:17:18.087 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:18.087+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:18.144 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:18.144 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:18.150 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:18.147+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:18.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:18.250 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:18.248+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:18.350 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:18.350 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.188982619000001, 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:18.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.123727861000006, 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:18.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.726385729, 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:18.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.723018039999999, 2024-09-21T23:17:18.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:18.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:18.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.718908459999994, 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.370 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:18.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:18.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:18.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.715115538999996, 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.654956732000002, 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:17:18.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.651199661000007, 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.651172860000003, 2024-09-21T23:17:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.651146694999994, 2024-09-21T23:17:18.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:18.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.651118245000006, 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:18.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:18.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.651027537000004, 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.650995957999996, 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:18.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:18.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:18.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:18.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.650958267999997, 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.650927878000005, 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:18.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:18.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.650884461000004, 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:18.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:18.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.650849836000006, 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:18.419 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:18.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.647596336000007, 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:18.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.647561753999994, 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.428 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:18.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.431817355, 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:18.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.438 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:18.438 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:18.347+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:18.439 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:18.483 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:18.483+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:18.551 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:18.551+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:18.625 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:18.626 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:18.626 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:18.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:18.684 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:18.683+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:18.753 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:18.752+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:18.834 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:18.834 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:18.834 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:18.834 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:18.834 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:18.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:18.895 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:18.895+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:18.954 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:18.952+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.223083899999999, 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:18.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.223038643000002, 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:18.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.995 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222989020999997, 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:18.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:18.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:18.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222959326000002, 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:19.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222926397999998, 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.005 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:19.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.009 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222892102000003, 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222867596, 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.014 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:19.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:19.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222839606000001, 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:19.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:19.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:19.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222816578, 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:19.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.027 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222795068000003, 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:19.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:19.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222759527000001, 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.033 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:19.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.037 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222731889999999, 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:19.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:19.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:19.042 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222696726000002, 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:19.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:19.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:19.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222661694999999, 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222626310000003, 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.052 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:19.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:19.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222586128000003, 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:19.056 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:19.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:19.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222549000000001, 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:19.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222508925, 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:19.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:19.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:19.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:19.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222484381000001, 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.070 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:19.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.074 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 62.222446595999997, 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:19.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:19.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:19.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:19.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:19.079 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:19.080 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:19.081 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:19.076+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:19.181 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:19.176+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:19.257 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:19.257 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:19.257 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:19.257 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:19.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:19.282 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:19.280+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:19.330 INFO:teuthology.orchestra.run.smithi170.stdout:5099 punch from 0x6662c00 to 0x6670c00, (0xe000 bytes) 2024-09-21T23:17:19.346 INFO:teuthology.orchestra.run.smithi170.stdout:5100 write 0x644da00 thru 0x6456ca4 (0x92a5 bytes) 2024-09-21T23:17:19.346 INFO:teuthology.orchestra.run.smithi170.stdout:5101 read 0x57ae800 thru 0x57b273a (0x3f3b bytes) 2024-09-21T23:17:19.346 INFO:teuthology.orchestra.run.smithi170.stdout:5103 write 0x3619400 thru 0x3624b52 (0xb753 bytes) 2024-09-21T23:17:19.347 INFO:teuthology.orchestra.run.smithi170.stdout:5106 write 0x612d600 thru 0x6132e0d (0x580e bytes) 2024-09-21T23:17:19.349 INFO:teuthology.orchestra.run.smithi170.stdout:5107 write 0x5c89800 thru 0x5c8a54b (0xd4c bytes) 2024-09-21T23:17:19.349 INFO:teuthology.orchestra.run.smithi170.stdout:5112 clone 32 order 25 su 8388608 sc 15 2024-09-21T23:17:19.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:19.383 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:19.383+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:19.399 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:19.399 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:19.399 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:19.399 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:19.400 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:19.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:19.507 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:19.507+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:19.531 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:19.531 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:19.548 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:19.548 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:19.565 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:19.565 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:19.582 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:19.582 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:19.584 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:19.583+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:19.586 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:19.586 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:19.596 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:19.608 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:19.630 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:19.691 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:19.691+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:19.729 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:19.729 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:19.743 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:19.790 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:19.788+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:19.863 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:19.863 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:19.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:19.886 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:19.883+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:19.987 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:19.984+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:20.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:17:20.111 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:20.111 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:20.111 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:20.112 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:20.107+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:20.189 INFO:teuthology.orchestra.run.smithi170.stdout:flattening image image_client.0-clone31 2024-09-21T23:17:20.211 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:20.211+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:20.287 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:20.287 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:20.301 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:20.346 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:20.344+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:20.413 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:20.408+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:20.423 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:20.423 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:20.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:20.526 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:20.524+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:20.605 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:20.606 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:20.614 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:20.612+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:20.620 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:20.715 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:20.711+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:20.779 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-21T23:17:20.787 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:20.787 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:20.801 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:20.816 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:20.815+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:20.868 INFO:teuthology.orchestra.run.smithi170.stdout:5114 punch from 0x7027a00 to 0x702fa00, (0x8000 bytes) 2024-09-21T23:17:20.882 INFO:teuthology.orchestra.run.smithi170.stdout:5115 punch from 0x6cd5200 to 0x6cdf000, (0x9e00 bytes) 2024-09-21T23:17:20.916 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:20.912+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:20.926 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:20.926 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:20.933 INFO:teuthology.orchestra.run.smithi170.stdout:5116 write 0xaf0f600 thru 0xaf176bb (0x80bc bytes) 2024-09-21T23:17:20.934 INFO:teuthology.orchestra.run.smithi170.stdout:5118 write 0x3a19200 thru 0x3a1ec9e (0x5a9f bytes) 2024-09-21T23:17:20.936 INFO:teuthology.orchestra.run.smithi170.stdout:5119 write 0x8070e00 thru 0x807cdad (0xbfae bytes) 2024-09-21T23:17:20.937 INFO:teuthology.orchestra.run.smithi170.stdout:5120 read 0x792cc00 thru 0x793afd6 (0xe3d7 bytes) 2024-09-21T23:17:20.937 INFO:teuthology.orchestra.run.smithi170.stdout:5121 punch from 0x7cb1e00 to 0x7cc0000, (0xe200 bytes) 2024-09-21T23:17:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:5123 punch from 0x514ae00 to 0x5157a00, (0xcc00 bytes) 2024-09-21T23:17:20.941 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:20.999 INFO:teuthology.orchestra.run.smithi170.stdout:5124 trunc from 0xaf17800 to 0x4dde400 2024-09-21T23:17:21.018 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:21.015+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:21.030 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:17:21.091 INFO:teuthology.orchestra.run.smithi170.stdout:5125 write 0x5722a00 thru 0x57322f6 (0xf8f7 bytes) 2024-09-21T23:17:21.092 INFO:teuthology.orchestra.run.smithi170.stdout:5127 read 0x4ad7400 thru 0x4adf85f (0x8460 bytes) 2024-09-21T23:17:21.096 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:21.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:17:21.117 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:21.117 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:21.118 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:21.116+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:21.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:21.156 INFO:teuthology.orchestra.run.smithi170.stdout:5128 write 0xce4c800 thru 0xce5154c (0x4d4d bytes) 2024-09-21T23:17:21.161 INFO:teuthology.orchestra.run.smithi170.stdout:5129 read 0xc4c6200 thru 0xc4cff72 (0x9d73 bytes) 2024-09-21T23:17:21.162 INFO:teuthology.orchestra.run.smithi170.stdout:5131 punch from 0x967400 to 0x96ba00, (0x4600 bytes) 2024-09-21T23:17:21.163 INFO:teuthology.orchestra.run.smithi170.stdout:5136 read 0xbf96800 thru 0xbf9ce01 (0x6602 bytes) 2024-09-21T23:17:21.163 INFO:teuthology.orchestra.run.smithi170.stdout:5141 read 0x2ab9600 thru 0x2abb749 (0x214a bytes) 2024-09-21T23:17:21.164 INFO:teuthology.orchestra.run.smithi170.stdout:5142 read 0x5a88000 thru 0x5a948b2 (0xc8b3 bytes) 2024-09-21T23:17:21.164 INFO:teuthology.orchestra.run.smithi170.stdout:5144 punch from 0xa125400 to 0xa126200, (0xe00 bytes) 2024-09-21T23:17:21.164 INFO:teuthology.orchestra.run.smithi170.stdout:5145 read 0x15cc600 thru 0x15cde14 (0x1815 bytes) 2024-09-21T23:17:21.175 INFO:teuthology.orchestra.run.smithi170.stdout:5146 trunc from 0xce51600 to 0x813aa00 2024-09-21T23:17:21.204 INFO:teuthology.orchestra.run.smithi170.stdout:5151 write 0x60e7e00 thru 0x60ec157 (0x4358 bytes) 2024-09-21T23:17:21.218 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:21.215+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:21.251 INFO:teuthology.orchestra.run.smithi170.stdout:5153 trunc from 0x813aa00 to 0x2427600 2024-09-21T23:17:21.285 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:21.285 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:21.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:21.345 INFO:teuthology.orchestra.run.smithi170.stdout:5154 read 0x64ce00 thru 0x65cd27 (0xff28 bytes) 2024-09-21T23:17:21.347 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:21.347+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:21.360 INFO:teuthology.orchestra.run.smithi170.stdout:5155 trunc from 0x2427600 to 0x6061200 2024-09-21T23:17:21.384 INFO:teuthology.orchestra.run.smithi170.stdout:5157 read 0x46d6200 thru 0x46e1c73 (0xba74 bytes) 2024-09-21T23:17:21.421 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:21.419+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:21.438 INFO:teuthology.orchestra.run.smithi170.stdout:5158 write 0xd802e00 thru 0xd81291c (0xfb1d bytes) 2024-09-21T23:17:21.440 INFO:teuthology.orchestra.run.smithi170.stdout:5159 punch from 0x6b65600 to 0x6b6e200, (0x8c00 bytes) 2024-09-21T23:17:21.441 INFO:teuthology.orchestra.run.smithi170.stdout:5161 punch from 0x7203e00 to 0x7208200, (0x4400 bytes) 2024-09-21T23:17:21.463 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:21.463 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:21.463 INFO:teuthology.orchestra.run.smithi170.stdout:5162 trunc from 0xd812a00 to 0x1c59e00 2024-09-21T23:17:21.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:21.522 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:21.520+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:21.568 INFO:teuthology.orchestra.run.smithi170.stdout:5165 write 0x4c94e00 thru 0x4ca33c8 (0xe5c9 bytes) 2024-09-21T23:17:21.623 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:21.620+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:21.639 INFO:teuthology.orchestra.run.smithi170.stdout:5166 write 0xe749800 thru 0xe7553df (0xbbe0 bytes) 2024-09-21T23:17:21.641 INFO:teuthology.orchestra.run.smithi170.stdout:5167 read 0x1e13e00 thru 0x1e187c5 (0x49c6 bytes) 2024-09-21T23:17:21.642 INFO:teuthology.orchestra.run.smithi170.stdout:5170 write 0xa85fc00 thru 0xa860967 (0xd68 bytes) 2024-09-21T23:17:21.642 INFO:teuthology.orchestra.run.smithi170.stdout:5173 punch from 0x5349000 to 0x534f400, (0x6400 bytes) 2024-09-21T23:17:21.643 INFO:teuthology.orchestra.run.smithi170.stdout:5174 punch from 0x3473600 to 0x347c400, (0x8e00 bytes) 2024-09-21T23:17:21.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:21.654 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:21.654 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:21.661 INFO:teuthology.orchestra.run.smithi170.stdout:5175 trunc from 0xe755400 to 0x79d1000 2024-09-21T23:17:21.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:21.693 INFO:teuthology.orchestra.run.smithi170.stdout:5176 punch from 0x43cce00 to 0x43d6200, (0x9400 bytes) 2024-09-21T23:17:21.694 INFO:teuthology.orchestra.run.smithi170.stdout:5179 read 0x1a1200 thru 0x1ae77d (0xd57e bytes) 2024-09-21T23:17:21.697 INFO:teuthology.orchestra.run.smithi170.stdout:5180 read 0x340da00 thru 0x341b3d0 (0xd9d1 bytes) 2024-09-21T23:17:21.724 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:21.720+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:21.727 INFO:teuthology.orchestra.run.smithi170.stdout:5181 write 0x81cb200 thru 0x81cf29b (0x409c bytes) 2024-09-21T23:17:21.771 INFO:teuthology.orchestra.run.smithi170.stdout:5182 write 0xd264a00 thru 0xd26ba46 (0x7047 bytes) 2024-09-21T23:17:21.774 INFO:teuthology.orchestra.run.smithi170.stdout:5183 write 0x167bc00 thru 0x167fc0b (0x400c bytes) 2024-09-21T23:17:21.776 INFO:teuthology.orchestra.run.smithi170.stdout:5184 write 0x635c000 thru 0x6368470 (0xc471 bytes) 2024-09-21T23:17:21.776 INFO:teuthology.orchestra.run.smithi170.stdout:5190 read 0x703c00 thru 0x707cd0 (0x40d1 bytes) 2024-09-21T23:17:21.777 INFO:teuthology.orchestra.run.smithi170.stdout:5192 read 0x39b5400 thru 0x39bd1dc (0x7ddd bytes) 2024-09-21T23:17:21.811 INFO:teuthology.orchestra.run.smithi170.stdout:5193 write 0xde42800 thru 0xde4e7d9 (0xbfda bytes) 2024-09-21T23:17:21.812 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:21.812 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:21.815 INFO:teuthology.orchestra.run.smithi170.stdout:5194 write 0x6904600 thru 0x690f35c (0xad5d bytes) 2024-09-21T23:17:21.817 INFO:teuthology.orchestra.run.smithi170.stdout:5197 punch from 0xc410e00 to 0xc418e00, (0x8000 bytes) 2024-09-21T23:17:21.818 INFO:teuthology.orchestra.run.smithi170.stdout:5198 read 0xaa9da00 thru 0xaaa9181 (0xb782 bytes) 2024-09-21T23:17:21.818 INFO:teuthology.orchestra.run.smithi170.stdout:5200 read 0x79e4c00 thru 0x79f12d1 (0xc6d2 bytes) 2024-09-21T23:17:21.819 INFO:teuthology.orchestra.run.smithi170.stdout:5202 punch from 0x613de00 to 0x614bc00, (0xde00 bytes) 2024-09-21T23:17:21.819 INFO:teuthology.orchestra.run.smithi170.stdout:5203 read 0xc595c00 thru 0xc5a1bd9 (0xbfda bytes) 2024-09-21T23:17:21.821 INFO:teuthology.orchestra.run.smithi170.stdout:5204 read 0x9186c00 thru 0x9189bd3 (0x2fd4 bytes) 2024-09-21T23:17:21.825 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:21.824+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:21.830 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:21.843 INFO:teuthology.orchestra.run.smithi170.stdout:5205 trunc from 0xde4e800 to 0x174d800 2024-09-21T23:17:21.845 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:21.845 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:21.847 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:21.847 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:21.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:21.870 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:21.870 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:21.908 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:21.908 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:21.926 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:21.924+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:21.989 INFO:teuthology.orchestra.run.smithi170.stdout:5206 write 0x8442800 thru 0x844ea32 (0xc233 bytes) 2024-09-21T23:17:21.989 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:21.990 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:21.990 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:21.990 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:21.990 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:21.992 INFO:teuthology.orchestra.run.smithi170.stdout:5208 write 0x1b3e600 thru 0x1b4118b (0x2b8c bytes) 2024-09-21T23:17:21.992 INFO:teuthology.orchestra.run.smithi170.stdout:5210 read 0xbdd200 thru 0xbebc83 (0xea84 bytes) 2024-09-21T23:17:21.998 INFO:teuthology.orchestra.run.smithi170.stdout:5213 trunc from 0x844ec00 to 0x685f400 2024-09-21T23:17:22.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:22.027 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:22.024+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:22.036 INFO:teuthology.orchestra.run.smithi170.stdout:5216 read 0xe61a00 thru 0xe6253c (0xb3d bytes) 2024-09-21T23:17:22.038 INFO:teuthology.orchestra.run.smithi170.stdout:5223 read 0x1d19c00 thru 0x1d20b0e (0x6f0f bytes) 2024-09-21T23:17:22.040 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:17:22.040 INFO:teuthology.orchestra.run.smithi170.stdout:5224 read 0x612a000 thru 0x6131002 (0x7003 bytes) 2024-09-21T23:17:22.041 INFO:teuthology.orchestra.run.smithi170.stdout:5225 read 0x55a0a00 thru 0x55aaa1b (0xa01c bytes) 2024-09-21T23:17:22.041 INFO:teuthology.orchestra.run.smithi170.stdout:5227 read 0x3291200 thru 0x32974e0 (0x62e1 bytes) 2024-09-21T23:17:22.041 INFO:teuthology.orchestra.run.smithi170.stdout:5228 read 0x4cb600 thru 0x4d7459 (0xbe5a bytes) 2024-09-21T23:17:22.049 INFO:teuthology.orchestra.run.smithi170.stdout:5231 punch from 0x56ec400 to 0x56efe00, (0x3a00 bytes) 2024-09-21T23:17:22.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:17:22.089 INFO:teuthology.orchestra.run.smithi170.stdout:5233 write 0xdb96600 thru 0xdb9d718 (0x7119 bytes) 2024-09-21T23:17:22.092 INFO:teuthology.orchestra.run.smithi170.stdout:5234 read 0xb12be00 thru 0xb1390e1 (0xd2e2 bytes) 2024-09-21T23:17:22.100 INFO:teuthology.orchestra.run.smithi170.stdout:5235 trunc from 0xdb9d800 to 0xab3fc00 2024-09-21T23:17:22.127 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:22.124+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:22.133 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:22.133 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:22.137 INFO:teuthology.orchestra.run.smithi170.stdout:5237 write 0x7834000 thru 0x7836a4f (0x2a50 bytes) 2024-09-21T23:17:22.165 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405637017999993, 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405591760999997, 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405542139000005, 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405512443999996, 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405479516, 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405445220000004, 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405420714000002, 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405392723999995, 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:22.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:22.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405369695999994, 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:22.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:22.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405348185999998, 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:22.207 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:22.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:22.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405312644999995, 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:22.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405285008000007, 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:22.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.221 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405249843999997, 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405214813000001, 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:22.230 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405179427999997, 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405139246000005, 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:22.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:22.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405102118000002, 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405062043000001, 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:22.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:22.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.405037499000002, 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.404999713999999, 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.258 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:22.259 INFO:teuthology.orchestra.run.smithi170.stdout:5239 write 0xdb6c000 thru 0xdb6daab (0x1aac bytes) 2024-09-21T23:17:22.259 INFO:teuthology.orchestra.run.smithi170.stdout:5240 read 0x8b49c00 thru 0x8b56617 (0xca18 bytes) 2024-09-21T23:17:22.259 INFO:teuthology.orchestra.run.smithi170.stdout:5241 punch from 0x5f1ba00 to 0x5f26600, (0xac00 bytes) 2024-09-21T23:17:22.259 INFO:teuthology.orchestra.run.smithi170.stdout:5242 read 0xa213a00 thru 0xa21c8e0 (0x8ee1 bytes) 2024-09-21T23:17:22.260 INFO:teuthology.orchestra.run.smithi170.stdout:5243 clone 33 order 25 su 16777216 sc 15 2024-09-21T23:17:22.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:22.260 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:22.255+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:22.360 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:22.359+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:22.428 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:22.428 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:22.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:22.483 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:22.483+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:22.562 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:22.559+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:22.608 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:22.609 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:22.623 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:22.671 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:22.671+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:22.764 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:22.760+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:22.802 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:22.802 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:22.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87448040699999, 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87345956, 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:22.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:22.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87339988700001, 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87330991100001, 2024-09-21T23:17:22.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87326000100001, 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87320386099999, 2024-09-21T23:17:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87315007799999, 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.873117802, 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87308752200001, 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.873057866, 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87302789899999, 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87295007, 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87290236299999, 2024-09-21T23:17:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.872872167, 2024-09-21T23:17:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87284225000001, 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.87281245, 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.872782517, 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.872717554, 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.872687078, 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.872657914, 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:22.901 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:22.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:22.902 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:22.900+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:23.003 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:23.000+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:23.033 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:23.033 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:23.033 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:23.033 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:23.046 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:17:23.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:17:23.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:17:23.126 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:23.123+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:23.177 INFO:teuthology.orchestra.run.smithi170.stdout:leaving image image_client.0-clone32 intact 2024-09-21T23:17:23.204 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:23.200+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:23.215 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:23.216 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:23.229 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:23.322 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:23.320+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:23.403 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 157.18060046599999, 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:23.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:23.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.66819518299999, 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.410 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:23.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.66813028000001, 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.421 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.66798384099999, 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:23.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:23.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:23.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:23.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:23.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.66791472599999, 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.427 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.667302401, 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:23.432 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.667097996, 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:23.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:23.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:23.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.66705521, 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.443 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:23.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.448 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27609692199999, 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27601041200001, 2024-09-21T23:17:23.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:23.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:23.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27593352299999, 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:23.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27572868300001, 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:23.466 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.471 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.275640401, 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:23.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:23.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27550957299999, 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27546391999999, 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.483 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:23.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:23.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:23.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:23.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.488 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.275425493, 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:23.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:23.494 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27529603100001, 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:23.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:23.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:23.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:23.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.275274389, 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:23.505 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27525434500001, 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 156.27517035299999, 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:23.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:23.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.517 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.517 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:23.518 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:23.518 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:23.516+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:23.619 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:23.615+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:23.695 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:23.695 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:23.695 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:23.695 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:23.709 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:23.720 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:23.719+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:23.810 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-21T23:17:23.820 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:23.819+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:23.829 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:23.830 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:23.844 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:23.920 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:23.916+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:23.968 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:23.968 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.01099883099999, 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.969 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:23.974 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.010731847, 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.979 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.398403302000006, 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:23.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.398360603, 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.984 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:23.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.398321983000002, 2024-09-21T23:17:23.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:23.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.992 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.398290110999994, 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.996 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.398256887000002, 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.997 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:23.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.398221812000003, 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.001 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:24.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.398176853999999, 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.397724787000001, 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.010 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:24.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:24.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.397010308000006, 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:24.014 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:24.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:24.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.396969237999997, 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.019 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:24.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:24.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:24.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.396512119999997, 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:24.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:24.027 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.396275441, 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:24.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:24.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.395859860000002, 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:24.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:24.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.395182199999994, 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.036 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:24.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.394690910999998, 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:24.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:24.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.394434012999994, 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.045 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:17:24.049 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.685242914, 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.053 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.682364213, 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:17:24.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:24.059 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:24.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:24.061 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:24.056+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:17:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:17:24.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:17:24.152 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:24.179 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:24.179+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:24.232 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:24.233 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:24.233 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:24.233 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:24.247 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:24.263 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:24.263+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:24.337 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:24.337 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:24.358 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:24.358 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:24.364 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:24.363+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:24.372 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:24.372 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:24.372 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:24.372 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:24.372 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:24.377 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:24.377 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:24.387 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:24.398 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:24.398 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:24.418 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:24.418 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:24.437 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:24.438 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:24.495 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:24.495+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:24.514 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:24.514 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:24.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:24.565 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:24.564+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:24.646 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:24.646 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:24.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:24.666 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:24.664+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:24.767 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:24.767+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:24.783 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:24.783 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:24.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:24.868 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:24.864+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:24.956 INFO:teuthology.orchestra.run.smithi170.stdout:5246 read 0x5d26e00 thru 0x5d2c33b (0x553c bytes) 2024-09-21T23:17:24.958 INFO:teuthology.orchestra.run.smithi170.stdout:5247 punch from 0x4448600 to 0x444de00, (0x5800 bytes) 2024-09-21T23:17:24.966 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:24.966 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:24.969 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:24.967+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:24.975 INFO:teuthology.orchestra.run.smithi170.stdout:5249 write 0x6d7b400 thru 0x6d7f467 (0x4068 bytes) 2024-09-21T23:17:24.975 INFO:teuthology.orchestra.run.smithi170.stdout:5250 write 0x11f7e00 thru 0x11fde6d (0x606e bytes) 2024-09-21T23:17:24.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:25.012 INFO:teuthology.orchestra.run.smithi170.stdout:5251 write 0xdd3f000 thru 0xdd4a924 (0xb925 bytes) 2024-09-21T23:17:25.013 INFO:teuthology.orchestra.run.smithi170.stdout:5255 write 0x38b1e00 thru 0x38b7eb2 (0x60b3 bytes) 2024-09-21T23:17:25.064 INFO:teuthology.orchestra.run.smithi170.stdout:5256 trunc from 0xdd4aa00 to 0x52d8c00 2024-09-21T23:17:25.065 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:17:25.069 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:25.064+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:25.089 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:17:25.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:17:25.147 INFO:teuthology.orchestra.run.smithi170.stdout:5257 read 0x706e00 thru 0x70cfea (0x61eb bytes) 2024-09-21T23:17:25.156 INFO:teuthology.orchestra.run.smithi170.stdout:5259 trunc from 0x52d8c00 to 0x6f57c00 2024-09-21T23:17:25.159 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:25.160 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:25.171 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:25.168+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:25.175 INFO:teuthology.orchestra.run.smithi170.stdout:5261 trunc from 0x6f57c00 to 0x5f44a00 2024-09-21T23:17:25.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:25.196 INFO:teuthology.orchestra.run.smithi170.stdout:5262 read 0x47f2e00 thru 0x47ff610 (0xc811 bytes) 2024-09-21T23:17:25.209 INFO:teuthology.orchestra.run.smithi170.stdout:5263 trunc from 0x5f44a00 to 0xa3e7200 2024-09-21T23:17:25.227 INFO:teuthology.orchestra.run.smithi170.stdout:5264 read 0x83a8000 thru 0x83b57d4 (0xd7d5 bytes) 2024-09-21T23:17:25.230 INFO:teuthology.orchestra.run.smithi170.stdout:5265 write 0xf09400 thru 0xf16f21 (0xdb22 bytes) 2024-09-21T23:17:25.242 INFO:teuthology.orchestra.run.smithi170.stdout:5266 trunc from 0xa3e7200 to 0x5520200 2024-09-21T23:17:25.270 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:25.267+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:25.295 INFO:teuthology.orchestra.run.smithi170.stdout:5267 punch from 0x33d600 to 0x340800, (0x3200 bytes) 2024-09-21T23:17:25.297 INFO:teuthology.orchestra.run.smithi170.stdout:5268 read 0x14fd600 thru 0x150182c (0x422d bytes) 2024-09-21T23:17:25.298 INFO:teuthology.orchestra.run.smithi170.stdout:5269 punch from 0x4f70a00 to 0x4f77a00, (0x7000 bytes) 2024-09-21T23:17:25.317 INFO:teuthology.orchestra.run.smithi170.stdout:5272 write 0x5631400 thru 0x563f0dc (0xdcdd bytes) 2024-09-21T23:17:25.319 INFO:teuthology.orchestra.run.smithi170.stdout:5274 punch from 0x3991600 to 0x399a600, (0x9000 bytes) 2024-09-21T23:17:25.334 INFO:teuthology.orchestra.run.smithi170.stdout:5277 trunc from 0x563f200 to 0xbf88600 2024-09-21T23:17:25.353 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:25.354 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:25.354 INFO:teuthology.orchestra.run.smithi170.stdout:5278 trunc from 0xbf88600 to 0xbf27a00 2024-09-21T23:17:25.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:25.376 INFO:teuthology.orchestra.run.smithi170.stdout:5279 punch from 0x580dc00 to 0x5819200, (0xb600 bytes) 2024-09-21T23:17:25.379 INFO:teuthology.orchestra.run.smithi170.stdout:5280 write 0xbc76600 thru 0xbc76dc8 (0x7c9 bytes) 2024-09-21T23:17:25.379 INFO:teuthology.orchestra.run.smithi170.stdout:5281 read 0x6f2fc00 thru 0x6f3dd92 (0xe193 bytes) 2024-09-21T23:17:25.379 INFO:teuthology.orchestra.run.smithi170.stdout:5283 read 0x439e000 thru 0x43aa577 (0xc578 bytes) 2024-09-21T23:17:25.380 INFO:teuthology.orchestra.run.smithi170.stdout:5285 read 0x739e600 thru 0x73a7f71 (0x9972 bytes) 2024-09-21T23:17:25.381 INFO:teuthology.orchestra.run.smithi170.stdout:5287 read 0x332c400 thru 0x333b2ab (0xeeac bytes) 2024-09-21T23:17:25.411 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:25.411+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:25.451 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:17:25.452 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:17:25.454 INFO:teuthology.orchestra.run.smithi170.stdout:5289 trunc from 0xbf27a00 to 0x7a9a00 2024-09-21T23:17:25.471 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-21T23:17:25.066661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":841560438,"num_objects":1478,"num_object_clones":404,"num_object_copies":2956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1478,"num_whiteouts":658,"num_read":271790,"num_read_kb":876391,"num_write":17190,"num_write_kb":163286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":23892960,"num_keys_recovered":282,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15480,"ondisk_log_size":15480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":410772,"kb_used_data":204340,"kb_used_omap":10,"kb_used_meta":206325,"kb_avail":377076588,"statfs":{"total":386547056640,"available":386126426112,"internally_reserved":0,"allocated":209244160,"data_stored":299587176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":211277428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60854988,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":2,"num_read":5147,"num_read_kb":20271,"num_write":310,"num_write_kb":4289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033239"},"pg_stats":[{"pgid":"3.1f","version":"118'396","reported_seq":11916,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.181243+0000","last_change":"2024-09-21T23:16:36.051323+0000","last_active":"2024-09-21T23:17:21.181243+0000","last_peered":"2024-09-21T23:17:21.181243+0000","last_clean":"2024-09-21T23:17:21.181243+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:17:21.181243+0000","last_undegraded":"2024-09-21T23:17:21.181243+0000","last_fullsized":"2024-09-21T23:17:21.181243+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:59.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":36711001,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":11588,"num_read_kb":18763,"num_write":505,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'388","reported_seq":4828,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119187+0000","last_change":"2024-09-21T23:16:36.049621+0000","last_active":"2024-09-21T23:17:24.119187+0000","last_peered":"2024-09-21T23:17:24.119187+0000","last_clean":"2024-09-21T23:17:24.119187+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:17:24.119187+0000","last_undegraded":"2024-09-21T23:17:24.119187+0000","last_fullsized":"2024-09-21T23:17:24.119187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:48.152663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":23831107,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":5015,"num_read_kb":25748,"num_write":347,"num_write_kb":6119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"119'365","reported_seq":6880,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.802243+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:17:24.802243+0000","last_peered":"2024-09-21T23:17:24.802243+0000","last_clean":"2024-09-21T23:17:24.802243+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:17:24.802243+0000","last_undegraded":"2024-09-21T23:17:24.802243+0000","last_fullsized":"2024-09-21T23:17:24.802243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:59:53.029662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":41049636,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":7314,"num_read_kb":36108,"num_write":298,"num_write_kb":17090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"120'610","reported_seq":10236,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109963+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:17:24.109963+0000","last_peered":"2024-09-21T23:17:24.109963+0000","last_clean":"2024-09-21T23:17:24.109963+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:17:24.109963+0000","last_undegraded":"2024-09-21T23:17:24.109963+0000","last_fullsized":"2024-09-21T23:17:24.109963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:51.950052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":34172496,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":13943,"num_read_kb":45578,"num_write":446,"num_write_kb":14152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"117'342","reported_seq":4330,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046363+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:17:21.046363+0000","last_peered":"2024-09-21T23:17:21.046363+0000","last_clean":"2024-09-21T23:17:21.046363+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:17:21.046363+0000","last_undegraded":"2024-09-21T23:17:21.046363+0000","last_fullsized":"2024-09-21T23:17:21.046363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:26:42.135032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":12618856,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":4860,"num_read_kb":19955,"num_write":353,"num_write_kb":1314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"116'397","reported_seq":8859,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.803680+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:17:24.803680+0000","last_peered":"2024-09-21T23:17:24.803680+0000","last_clean":"2024-09-21T23:17:24.803680+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:17:24.803680+0000","last_undegraded":"2024-09-21T23:17:24.803680+0000","last_fullsized":"2024-09-21T23:17:24.803680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:25.801948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":11755043,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":9956,"num_read_kb":24206,"num_write":300,"num_write_kb":1244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"118'307","reported_seq":4366,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.796644+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:17:24.796644+0000","last_peered":"2024-09-21T23:17:24.796644+0000","last_clean":"2024-09-21T23:17:24.796644+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:17:24.796644+0000","last_undegraded":"2024-09-21T23:17:24.796644+0000","last_fullsized":"2024-09-21T23:17:24.796644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:04:55.643072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":58400816,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":4186,"num_read_kb":61491,"num_write":253,"num_write_kb":30841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"114'520","reported_seq":17460,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046302+0000","last_change":"2024-09-21T23:16:36.046053+0000","last_active":"2024-09-21T23:17:21.046302+0000","last_peered":"2024-09-21T23:17:21.046302+0000","last_clean":"2024-09-21T23:17:21.046302+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:17:21.046302+0000","last_undegraded":"2024-09-21T23:17:21.046302+0000","last_fullsized":"2024-09-21T23:17:21.046302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:33:17.955292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":55256695,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":15979,"num_read_kb":49681,"num_write":645,"num_write_kb":11238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"116'326","reported_seq":13007,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046781+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:17:21.046781+0000","last_peered":"2024-09-21T23:17:21.046781+0000","last_clean":"2024-09-21T23:17:21.046781+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:17:21.046781+0000","last_undegraded":"2024-09-21T23:17:21.046781+0000","last_fullsized":"2024-09-21T23:17:21.046781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:30.059870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":25262205,"num_objects":54,"num_object_clones":18,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":13051,"num_read_kb":33576,"num_write":279,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"114'472","reported_seq":5318,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109686+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:17:24.109686+0000","last_peered":"2024-09-21T23:17:24.109686+0000","last_clean":"2024-09-21T23:17:24.109686+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:17:24.109686+0000","last_undegraded":"2024-09-21T23:17:24.109686+0000","last_fullsized":"2024-09-21T23:17:24.109686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:56.797583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":45419208,"num_objects":82,"num_object_clones":30,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":5313,"num_read_kb":21318,"num_write":539,"num_write_kb":3199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"118'289","reported_seq":3633,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046850+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:17:21.046850+0000","last_peered":"2024-09-21T23:17:21.046850+0000","last_clean":"2024-09-21T23:17:21.046850+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:17:21.046850+0000","last_undegraded":"2024-09-21T23:17:21.046850+0000","last_fullsized":"2024-09-21T23:17:21.046850+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:44:34.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31808035,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":3614,"num_read_kb":26675,"num_write":336,"num_write_kb":3532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"120'777","reported_seq":7111,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504530+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:17:24.504530+0000","last_peered":"2024-09-21T23:17:24.504530+0000","last_clean":"2024-09-21T23:17:24.504530+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:17:24.504530+0000","last_undegraded":"2024-09-21T23:17:24.504530+0000","last_fullsized":"2024-09-21T23:17:24.504530+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.096425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31251615,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":10180,"num_read_kb":34734,"num_write":970,"num_write_kb":12439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":207,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504686+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:17:24.504686+0000","last_peered":"2024-09-21T23:17:24.504686+0000","last_clean":"2024-09-21T23:17:24.504686+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:17:24.504686+0000","last_undegraded":"2024-09-21T23:17:24.504686+0000","last_fullsized":"2024-09-21T23:17:24.504686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"117'655","reported_seq":19365,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046623+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:17:21.046623+0000","last_peered":"2024-09-21T23:17:21.046623+0000","last_clean":"2024-09-21T23:17:21.046623+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:17:21.046623+0000","last_undegraded":"2024-09-21T23:17:21.046623+0000","last_fullsized":"2024-09-21T23:17:21.046623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:38:53.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":45253731,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":20359,"num_read_kb":32292,"num_write":975,"num_write_kb":4463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119246+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:17:24.119246+0000","last_peered":"2024-09-21T23:17:24.119246+0000","last_clean":"2024-09-21T23:17:24.119246+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:17:24.119246+0000","last_undegraded":"2024-09-21T23:17:24.119246+0000","last_fullsized":"2024-09-21T23:17:24.119246+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"120'1568","reported_seq":7426,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119123+0000","last_change":"2024-09-21T23:16:36.052892+0000","last_active":"2024-09-21T23:17:24.119123+0000","last_peered":"2024-09-21T23:17:24.119123+0000","last_clean":"2024-09-21T23:17:24.119123+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:17:24.119123+0000","last_undegraded":"2024-09-21T23:17:24.119123+0000","last_fullsized":"2024-09-21T23:17:24.119123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1568,"log_dups_size":0,"ondisk_log_size":1568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:34.240363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":11890144,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":12590,"num_read_kb":39636,"num_write":2731,"num_write_kb":6479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046741+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:17:21.046741+0000","last_peered":"2024-09-21T23:17:21.046741+0000","last_clean":"2024-09-21T23:17:21.046741+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:17:21.046741+0000","last_undegraded":"2024-09-21T23:17:21.046741+0000","last_fullsized":"2024-09-21T23:17:21.046741+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"118'345","reported_seq":1624,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119119+0000","last_change":"2024-09-21T23:16:36.050246+0000","last_active":"2024-09-21T23:17:24.119119+0000","last_peered":"2024-09-21T23:17:24.119119+0000","last_clean":"2024-09-21T23:17:24.119119+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:17:24.119119+0000","last_undegraded":"2024-09-21T23:17:24.119119+0000","last_fullsized":"2024-09-21T23:17:24.119119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:19.307344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":9744158,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":1983,"num_read_kb":3778,"num_write":375,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":110,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118949+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:17:24.118949+0000","last_peered":"2024-09-21T23:17:24.118949+0000","last_clean":"2024-09-21T23:17:24.118949+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:17:24.118949+0000","last_undegraded":"2024-09-21T23:17:24.118949+0000","last_fullsized":"2024-09-21T23:17:24.118949+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"116'438","reported_seq":9534,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.793966+0000","last_change":"2024-09-21T23:16:54.784043+0000","last_active":"2024-09-21T23:17:24.793966+0000","last_peered":"2024-09-21T23:17:24.793966+0000","last_clean":"2024-09-21T23:17:24.793966+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:17:24.793966+0000","last_undegraded":"2024-09-21T23:17:24.793966+0000","last_fullsized":"2024-09-21T23:17:24.793966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:53.402241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":47775507,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":10452,"num_read_kb":68452,"num_write":527,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"115'527","reported_seq":5808,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504467+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:17:24.504467+0000","last_peered":"2024-09-21T23:17:24.504467+0000","last_clean":"2024-09-21T23:17:24.504467+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:17:24.504467+0000","last_undegraded":"2024-09-21T23:17:24.504467+0000","last_fullsized":"2024-09-21T23:17:24.504467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:26:22.549233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":15034029,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":7460,"num_read_kb":22133,"num_write":519,"num_write_kb":2170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":246,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109311+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:17:24.109311+0000","last_peered":"2024-09-21T23:17:24.109311+0000","last_clean":"2024-09-21T23:17:24.109311+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:17:24.109311+0000","last_undegraded":"2024-09-21T23:17:24.109311+0000","last_fullsized":"2024-09-21T23:17:24.109311+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"118'282","reported_seq":5530,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118954+0000","last_change":"2024-09-21T23:16:36.051625+0000","last_active":"2024-09-21T23:17:24.118954+0000","last_peered":"2024-09-21T23:17:24.118954+0000","last_clean":"2024-09-21T23:17:24.118954+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:17:24.118954+0000","last_undegraded":"2024-09-21T23:17:24.118954+0000","last_fullsized":"2024-09-21T23:17:24.118954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:05:03.287242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":22986356,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":5213,"num_read_kb":21622,"num_write":238,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":211,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118705+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:17:24.118705+0000","last_peered":"2024-09-21T23:17:24.118705+0000","last_clean":"2024-09-21T23:17:24.118705+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:17:24.118705+0000","last_undegraded":"2024-09-21T23:17:24.118705+0000","last_fullsized":"2024-09-21T23:17:24.118705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"118'326","reported_seq":3919,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.052928+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:17:21.052928+0000","last_peered":"2024-09-21T23:17:21.052928+0000","last_clean":"2024-09-21T23:17:21.052928+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:17:21.052928+0000","last_undegraded":"2024-09-21T23:17:21.052928+0000","last_fullsized":"2024-09-21T23:17:21.052928+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:51.939761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":15456879,"num_objects":50,"num_object_clones":17,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":3487,"num_read_kb":14347,"num_write":317,"num_write_kb":5471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":110,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118814+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:17:24.118814+0000","last_peered":"2024-09-21T23:17:24.118814+0000","last_clean":"2024-09-21T23:17:24.118814+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:17:24.118814+0000","last_undegraded":"2024-09-21T23:17:24.118814+0000","last_fullsized":"2024-09-21T23:17:24.118814+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"114'430","reported_seq":12786,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118903+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:17:24.118903+0000","last_peered":"2024-09-21T23:17:24.118903+0000","last_clean":"2024-09-21T23:17:24.118903+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:17:24.118903+0000","last_undegraded":"2024-09-21T23:17:24.118903+0000","last_fullsized":"2024-09-21T23:17:24.118903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:29:34.953152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":20858544,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":13983,"num_read_kb":32782,"num_write":379,"num_write_kb":3530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":233,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109551+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:17:24.109551+0000","last_peered":"2024-09-21T23:17:24.109551+0000","last_clean":"2024-09-21T23:17:24.109551+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:17:24.109551+0000","last_undegraded":"2024-09-21T23:17:24.109551+0000","last_fullsized":"2024-09-21T23:17:24.109551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":316,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046151+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:17:21.046151+0000","last_peered":"2024-09-21T23:17:21.046151+0000","last_clean":"2024-09-21T23:17:21.046151+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:17:21.046151+0000","last_undegraded":"2024-09-21T23:17:21.046151+0000","last_fullsized":"2024-09-21T23:17:21.046151+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"118'804","reported_seq":11696,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118818+0000","last_change":"2024-09-21T23:16:36.047835+0000","last_active":"2024-09-21T23:17:24.118818+0000","last_peered":"2024-09-21T23:17:24.118818+0000","last_clean":"2024-09-21T23:17:24.118818+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:17:24.118818+0000","last_undegraded":"2024-09-21T23:17:24.118818+0000","last_fullsized":"2024-09-21T23:17:24.118818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:29:21.964511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":36265277,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":12688,"num_read_kb":24820,"num_write":697,"num_write_kb":2932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"115'446","reported_seq":4060,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504450+0000","last_change":"2024-09-21T23:16:36.050207+0000","last_active":"2024-09-21T23:17:24.504450+0000","last_peered":"2024-09-21T23:17:24.504450+0000","last_clean":"2024-09-21T23:17:24.504450+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:17:24.504450+0000","last_undegraded":"2024-09-21T23:17:24.504450+0000","last_fullsized":"2024-09-21T23:17:24.504450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:21:57.161673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":15650023,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":5276,"num_read_kb":11757,"num_write":361,"num_write_kb":1633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"120'247","reported_seq":3049,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504263+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:17:24.504263+0000","last_peered":"2024-09-21T23:17:24.504263+0000","last_clean":"2024-09-21T23:17:24.504263+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:17:24.504263+0000","last_undegraded":"2024-09-21T23:17:24.504263+0000","last_fullsized":"2024-09-21T23:17:24.504263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:21.512989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021264,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":2517,"num_read_kb":14502,"num_write":225,"num_write_kb":1186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"115'644","reported_seq":7374,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118962+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:17:24.118962+0000","last_peered":"2024-09-21T23:17:24.118962+0000","last_clean":"2024-09-21T23:17:24.118962+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:17:24.118962+0000","last_undegraded":"2024-09-21T23:17:24.118962+0000","last_fullsized":"2024-09-21T23:17:24.118962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:54.665875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":16158269,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":9758,"num_read_kb":19499,"num_write":598,"num_write_kb":1859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"121'412","reported_seq":3459,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.818146+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:17:24.818146+0000","last_peered":"2024-09-21T23:17:24.818146+0000","last_clean":"2024-09-21T23:17:24.818146+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:17:24.818146+0000","last_undegraded":"2024-09-21T23:17:24.818146+0000","last_fullsized":"2024-09-21T23:17:24.818146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:49:11.993843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":22625325,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":4292,"num_read_kb":8336,"num_write":340,"num_write_kb":2057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'335","reported_seq":3944,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504499+0000","last_change":"2024-09-21T23:16:36.052262+0000","last_active":"2024-09-21T23:17:24.504499+0000","last_peered":"2024-09-21T23:17:24.504499+0000","last_clean":"2024-09-21T23:17:24.504499+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:17:24.504499+0000","last_undegraded":"2024-09-21T23:17:24.504499+0000","last_fullsized":"2024-09-21T23:17:24.504499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:29:12.992703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7152434,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":4138,"num_read_kb":25418,"num_write":433,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"114'329","reported_seq":3248,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109749+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:17:24.109749+0000","last_peered":"2024-09-21T23:17:24.109749+0000","last_clean":"2024-09-21T23:17:24.109749+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:17:24.109749+0000","last_undegraded":"2024-09-21T23:17:24.109749+0000","last_fullsized":"2024-09-21T23:17:24.109749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:13:12.050563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":10085470,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":3880,"num_read_kb":11988,"num_write":322,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"117'360","reported_seq":4984,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.790030+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:17:24.790030+0000","last_peered":"2024-09-21T23:17:24.790030+0000","last_clean":"2024-09-21T23:17:24.790030+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:17:24.790030+0000","last_undegraded":"2024-09-21T23:17:24.790030+0000","last_fullsized":"2024-09-21T23:17:24.790030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:40.183577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":30326889,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":6436,"num_read_kb":19633,"num_write":328,"num_write_kb":1506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"118'590","reported_seq":3837,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109357+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:17:24.109357+0000","last_peered":"2024-09-21T23:17:24.109357+0000","last_clean":"2024-09-21T23:17:24.109357+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:17:24.109357+0000","last_undegraded":"2024-09-21T23:17:24.109357+0000","last_fullsized":"2024-09-21T23:17:24.109357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:33:45.358491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":17421417,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":4496,"num_read_kb":17102,"num_write":796,"num_write_kb":3825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"113'444","reported_seq":4638,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504341+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:17:24.504341+0000","last_peered":"2024-09-21T23:17:24.504341+0000","last_clean":"2024-09-21T23:17:24.504341+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:17:24.504341+0000","last_undegraded":"2024-09-21T23:17:24.504341+0000","last_fullsized":"2024-09-21T23:17:24.504341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:48:36.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19368523,"num_objects":50,"num_object_clones":19,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":25,"num_read":5120,"num_read_kb":24578,"num_write":594,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"117'629","reported_seq":20883,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046340+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:17:21.046340+0000","last_peered":"2024-09-21T23:17:21.046340+0000","last_clean":"2024-09-21T23:17:21.046340+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:17:21.046340+0000","last_undegraded":"2024-09-21T23:17:21.046340+0000","last_fullsized":"2024-09-21T23:17:21.046340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:18.868862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":37089895,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":23507,"num_read_kb":44206,"num_write":602,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"120'360","reported_seq":9258,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109336+0000","last_change":"2024-09-21T23:16:36.050076+0000","last_active":"2024-09-21T23:17:24.109336+0000","last_peered":"2024-09-21T23:17:24.109336+0000","last_clean":"2024-09-21T23:17:24.109336+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:17:24.109336+0000","last_undegraded":"2024-09-21T23:17:24.109336+0000","last_fullsized":"2024-09-21T23:17:24.109336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:26.136287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269204,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":9040,"num_read_kb":21577,"num_write":401,"num_write_kb":1948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":840970051,"num_objects":1474,"num_object_clones":404,"num_object_copies":2948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1474,"num_whiteouts":658,"num_read":271674,"num_read_kb":876291,"num_write":17029,"num_write_kb":160868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2034,"num_bytes_recovered":22712186,"num_keys_recovered":282,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203010048,"data_stored":296640326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":15360,"ondisk_log_size":15360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106004,"kb_used_data":52600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94265836,"statfs":{"total":96636764160,"available":96528216064,"internally_reserved":0,"allocated":53862400,"data_stored":83031680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134148,"kb_used_data":76588,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94237692,"statfs":{"total":96636764160,"available":96499396608,"internally_reserved":0,"allocated":78426112,"data_stored":119567659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607589,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":120340,"kb_used_data":53048,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94251500,"statfs":{"total":96636764160,"available":96513536000,"internally_reserved":0,"allocated":54321152,"data_stored":70210533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005073,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50280,"kb_used_data":22104,"kb_used_omap":5,"kb_used_meta":28154,"kb_avail":94321560,"statfs":{"total":96636764160,"available":96585277440,"internally_reserved":0,"allocated":22634496,"data_stored":26777304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":28829989},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20819968,"data_stored":25774485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52301824,"data_stored":69034846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77004800,"data_stored":118982359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":52424704,"data_stored":82441420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:17:25.474 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:17:25.515 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:25.515+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:25.530 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:25.530 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:25.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:25.568 INFO:teuthology.orchestra.run.smithi170.stdout:5291 write 0x6e46600 thru 0x6e4eb31 (0x8532 bytes) 2024-09-21T23:17:25.587 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:25.587+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:25.622 INFO:teuthology.orchestra.run.smithi170.stdout:5292 write 0xe5a8c00 thru 0xe5b1245 (0x8646 bytes) 2024-09-21T23:17:25.624 INFO:teuthology.orchestra.run.smithi170.stdout:5296 read 0x9832c00 thru 0x983ad75 (0x8176 bytes) 2024-09-21T23:17:25.624 INFO:teuthology.orchestra.run.smithi170.stdout:5300 clone 34 order 20 su 524288 sc 12 2024-09-21T23:17:25.676 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:25.675+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:25.712 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:25.712 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:25.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:25.815 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:25.815+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:25.828 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:17:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:17:25.848 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-21T23:17:25.066661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":841560438,"num_objects":1478,"num_object_clones":404,"num_object_copies":2956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1478,"num_whiteouts":658,"num_read":271790,"num_read_kb":876391,"num_write":17190,"num_write_kb":163286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":23892960,"num_keys_recovered":282,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15480,"ondisk_log_size":15480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":410772,"kb_used_data":204340,"kb_used_omap":10,"kb_used_meta":206325,"kb_avail":377076588,"statfs":{"total":386547056640,"available":386126426112,"internally_reserved":0,"allocated":209244160,"data_stored":299587176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":211277428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60854988,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":2,"num_read":5147,"num_read_kb":20271,"num_write":310,"num_write_kb":4289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033239"},"pg_stats":[{"pgid":"3.1f","version":"118'396","reported_seq":11916,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.181243+0000","last_change":"2024-09-21T23:16:36.051323+0000","last_active":"2024-09-21T23:17:21.181243+0000","last_peered":"2024-09-21T23:17:21.181243+0000","last_clean":"2024-09-21T23:17:21.181243+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:17:21.181243+0000","last_undegraded":"2024-09-21T23:17:21.181243+0000","last_fullsized":"2024-09-21T23:17:21.181243+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:02:59.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":36711001,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":26,"num_read":11588,"num_read_kb":18763,"num_write":505,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'388","reported_seq":4828,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119187+0000","last_change":"2024-09-21T23:16:36.049621+0000","last_active":"2024-09-21T23:17:24.119187+0000","last_peered":"2024-09-21T23:17:24.119187+0000","last_clean":"2024-09-21T23:17:24.119187+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:17:24.119187+0000","last_undegraded":"2024-09-21T23:17:24.119187+0000","last_fullsized":"2024-09-21T23:17:24.119187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:17:48.152663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":23831107,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":5015,"num_read_kb":25748,"num_write":347,"num_write_kb":6119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"119'365","reported_seq":6880,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.802243+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:17:24.802243+0000","last_peered":"2024-09-21T23:17:24.802243+0000","last_clean":"2024-09-21T23:17:24.802243+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:17:24.802243+0000","last_undegraded":"2024-09-21T23:17:24.802243+0000","last_fullsized":"2024-09-21T23:17:24.802243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:59:53.029662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":41049636,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":7314,"num_read_kb":36108,"num_write":298,"num_write_kb":17090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"120'610","reported_seq":10236,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109963+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:17:24.109963+0000","last_peered":"2024-09-21T23:17:24.109963+0000","last_clean":"2024-09-21T23:17:24.109963+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:17:24.109963+0000","last_undegraded":"2024-09-21T23:17:24.109963+0000","last_fullsized":"2024-09-21T23:17:24.109963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:51.950052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":34172496,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":13943,"num_read_kb":45578,"num_write":446,"num_write_kb":14152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"117'342","reported_seq":4330,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046363+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:17:21.046363+0000","last_peered":"2024-09-21T23:17:21.046363+0000","last_clean":"2024-09-21T23:17:21.046363+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:17:21.046363+0000","last_undegraded":"2024-09-21T23:17:21.046363+0000","last_fullsized":"2024-09-21T23:17:21.046363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:26:42.135032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":12618856,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":4860,"num_read_kb":19955,"num_write":353,"num_write_kb":1314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"116'397","reported_seq":8859,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.803680+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:17:24.803680+0000","last_peered":"2024-09-21T23:17:24.803680+0000","last_clean":"2024-09-21T23:17:24.803680+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:17:24.803680+0000","last_undegraded":"2024-09-21T23:17:24.803680+0000","last_fullsized":"2024-09-21T23:17:24.803680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:25.801948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":11755043,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":9956,"num_read_kb":24206,"num_write":300,"num_write_kb":1244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"118'307","reported_seq":4366,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.796644+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:17:24.796644+0000","last_peered":"2024-09-21T23:17:24.796644+0000","last_clean":"2024-09-21T23:17:24.796644+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:17:24.796644+0000","last_undegraded":"2024-09-21T23:17:24.796644+0000","last_fullsized":"2024-09-21T23:17:24.796644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:04:55.643072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":58400816,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":4186,"num_read_kb":61491,"num_write":253,"num_write_kb":30841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"114'520","reported_seq":17460,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046302+0000","last_change":"2024-09-21T23:16:36.046053+0000","last_active":"2024-09-21T23:17:21.046302+0000","last_peered":"2024-09-21T23:17:21.046302+0000","last_clean":"2024-09-21T23:17:21.046302+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:17:21.046302+0000","last_undegraded":"2024-09-21T23:17:21.046302+0000","last_fullsized":"2024-09-21T23:17:21.046302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:33:17.955292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":55256695,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":15979,"num_read_kb":49681,"num_write":645,"num_write_kb":11238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"116'326","reported_seq":13007,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046781+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:17:21.046781+0000","last_peered":"2024-09-21T23:17:21.046781+0000","last_clean":"2024-09-21T23:17:21.046781+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:17:21.046781+0000","last_undegraded":"2024-09-21T23:17:21.046781+0000","last_fullsized":"2024-09-21T23:17:21.046781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:30.059870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":25262205,"num_objects":54,"num_object_clones":18,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":13051,"num_read_kb":33576,"num_write":279,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"114'472","reported_seq":5318,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109686+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:17:24.109686+0000","last_peered":"2024-09-21T23:17:24.109686+0000","last_clean":"2024-09-21T23:17:24.109686+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:17:24.109686+0000","last_undegraded":"2024-09-21T23:17:24.109686+0000","last_fullsized":"2024-09-21T23:17:24.109686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:56.797583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":45419208,"num_objects":82,"num_object_clones":30,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":5313,"num_read_kb":21318,"num_write":539,"num_write_kb":3199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"118'289","reported_seq":3633,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046850+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:17:21.046850+0000","last_peered":"2024-09-21T23:17:21.046850+0000","last_clean":"2024-09-21T23:17:21.046850+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:17:21.046850+0000","last_undegraded":"2024-09-21T23:17:21.046850+0000","last_fullsized":"2024-09-21T23:17:21.046850+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:44:34.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31808035,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":3614,"num_read_kb":26675,"num_write":336,"num_write_kb":3532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"120'777","reported_seq":7111,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504530+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:17:24.504530+0000","last_peered":"2024-09-21T23:17:24.504530+0000","last_clean":"2024-09-21T23:17:24.504530+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:17:24.504530+0000","last_undegraded":"2024-09-21T23:17:24.504530+0000","last_fullsized":"2024-09-21T23:17:24.504530+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:10:02.096425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31251615,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":10180,"num_read_kb":34734,"num_write":970,"num_write_kb":12439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":207,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504686+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:17:24.504686+0000","last_peered":"2024-09-21T23:17:24.504686+0000","last_clean":"2024-09-21T23:17:24.504686+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:17:24.504686+0000","last_undegraded":"2024-09-21T23:17:24.504686+0000","last_fullsized":"2024-09-21T23:17:24.504686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"117'655","reported_seq":19365,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046623+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:17:21.046623+0000","last_peered":"2024-09-21T23:17:21.046623+0000","last_clean":"2024-09-21T23:17:21.046623+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:17:21.046623+0000","last_undegraded":"2024-09-21T23:17:21.046623+0000","last_fullsized":"2024-09-21T23:17:21.046623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:38:53.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":45253731,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":20359,"num_read_kb":32292,"num_write":975,"num_write_kb":4463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119246+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:17:24.119246+0000","last_peered":"2024-09-21T23:17:24.119246+0000","last_clean":"2024-09-21T23:17:24.119246+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:17:24.119246+0000","last_undegraded":"2024-09-21T23:17:24.119246+0000","last_fullsized":"2024-09-21T23:17:24.119246+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"120'1568","reported_seq":7426,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119123+0000","last_change":"2024-09-21T23:16:36.052892+0000","last_active":"2024-09-21T23:17:24.119123+0000","last_peered":"2024-09-21T23:17:24.119123+0000","last_clean":"2024-09-21T23:17:24.119123+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:17:24.119123+0000","last_undegraded":"2024-09-21T23:17:24.119123+0000","last_fullsized":"2024-09-21T23:17:24.119123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1568,"log_dups_size":0,"ondisk_log_size":1568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:34.240363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":11890144,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":12590,"num_read_kb":39636,"num_write":2731,"num_write_kb":6479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046741+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:17:21.046741+0000","last_peered":"2024-09-21T23:17:21.046741+0000","last_clean":"2024-09-21T23:17:21.046741+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:17:21.046741+0000","last_undegraded":"2024-09-21T23:17:21.046741+0000","last_fullsized":"2024-09-21T23:17:21.046741+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"118'345","reported_seq":1624,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.119119+0000","last_change":"2024-09-21T23:16:36.050246+0000","last_active":"2024-09-21T23:17:24.119119+0000","last_peered":"2024-09-21T23:17:24.119119+0000","last_clean":"2024-09-21T23:17:24.119119+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:17:24.119119+0000","last_undegraded":"2024-09-21T23:17:24.119119+0000","last_fullsized":"2024-09-21T23:17:24.119119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:19.307344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":9744158,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":1983,"num_read_kb":3778,"num_write":375,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":110,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118949+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:17:24.118949+0000","last_peered":"2024-09-21T23:17:24.118949+0000","last_clean":"2024-09-21T23:17:24.118949+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:17:24.118949+0000","last_undegraded":"2024-09-21T23:17:24.118949+0000","last_fullsized":"2024-09-21T23:17:24.118949+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"116'438","reported_seq":9534,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.793966+0000","last_change":"2024-09-21T23:16:54.784043+0000","last_active":"2024-09-21T23:17:24.793966+0000","last_peered":"2024-09-21T23:17:24.793966+0000","last_clean":"2024-09-21T23:17:24.793966+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:17:24.793966+0000","last_undegraded":"2024-09-21T23:17:24.793966+0000","last_fullsized":"2024-09-21T23:17:24.793966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:53.402241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":47775507,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":10452,"num_read_kb":68452,"num_write":527,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"115'527","reported_seq":5808,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504467+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:17:24.504467+0000","last_peered":"2024-09-21T23:17:24.504467+0000","last_clean":"2024-09-21T23:17:24.504467+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:17:24.504467+0000","last_undegraded":"2024-09-21T23:17:24.504467+0000","last_fullsized":"2024-09-21T23:17:24.504467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:26:22.549233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":15034029,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":7460,"num_read_kb":22133,"num_write":519,"num_write_kb":2170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":246,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109311+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:17:24.109311+0000","last_peered":"2024-09-21T23:17:24.109311+0000","last_clean":"2024-09-21T23:17:24.109311+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:17:24.109311+0000","last_undegraded":"2024-09-21T23:17:24.109311+0000","last_fullsized":"2024-09-21T23:17:24.109311+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"118'282","reported_seq":5530,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118954+0000","last_change":"2024-09-21T23:16:36.051625+0000","last_active":"2024-09-21T23:17:24.118954+0000","last_peered":"2024-09-21T23:17:24.118954+0000","last_clean":"2024-09-21T23:17:24.118954+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:17:24.118954+0000","last_undegraded":"2024-09-21T23:17:24.118954+0000","last_fullsized":"2024-09-21T23:17:24.118954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:05:03.287242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":22986356,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":5213,"num_read_kb":21622,"num_write":238,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":211,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118705+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:17:24.118705+0000","last_peered":"2024-09-21T23:17:24.118705+0000","last_clean":"2024-09-21T23:17:24.118705+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:17:24.118705+0000","last_undegraded":"2024-09-21T23:17:24.118705+0000","last_fullsized":"2024-09-21T23:17:24.118705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"118'326","reported_seq":3919,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.052928+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:17:21.052928+0000","last_peered":"2024-09-21T23:17:21.052928+0000","last_clean":"2024-09-21T23:17:21.052928+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:17:21.052928+0000","last_undegraded":"2024-09-21T23:17:21.052928+0000","last_fullsized":"2024-09-21T23:17:21.052928+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:51.939761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":15456879,"num_objects":50,"num_object_clones":17,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":3487,"num_read_kb":14347,"num_write":317,"num_write_kb":5471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":110,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118814+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:17:24.118814+0000","last_peered":"2024-09-21T23:17:24.118814+0000","last_clean":"2024-09-21T23:17:24.118814+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:17:24.118814+0000","last_undegraded":"2024-09-21T23:17:24.118814+0000","last_fullsized":"2024-09-21T23:17:24.118814+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"114'430","reported_seq":12786,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118903+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:17:24.118903+0000","last_peered":"2024-09-21T23:17:24.118903+0000","last_clean":"2024-09-21T23:17:24.118903+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:17:24.118903+0000","last_undegraded":"2024-09-21T23:17:24.118903+0000","last_fullsized":"2024-09-21T23:17:24.118903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:29:34.953152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":20858544,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":13983,"num_read_kb":32782,"num_write":379,"num_write_kb":3530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":233,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109551+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:17:24.109551+0000","last_peered":"2024-09-21T23:17:24.109551+0000","last_clean":"2024-09-21T23:17:24.109551+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:17:24.109551+0000","last_undegraded":"2024-09-21T23:17:24.109551+0000","last_fullsized":"2024-09-21T23:17:24.109551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":316,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046151+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:17:21.046151+0000","last_peered":"2024-09-21T23:17:21.046151+0000","last_clean":"2024-09-21T23:17:21.046151+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:17:21.046151+0000","last_undegraded":"2024-09-21T23:17:21.046151+0000","last_fullsized":"2024-09-21T23:17:21.046151+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"118'804","reported_seq":11696,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118818+0000","last_change":"2024-09-21T23:16:36.047835+0000","last_active":"2024-09-21T23:17:24.118818+0000","last_peered":"2024-09-21T23:17:24.118818+0000","last_clean":"2024-09-21T23:17:24.118818+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:17:24.118818+0000","last_undegraded":"2024-09-21T23:17:24.118818+0000","last_fullsized":"2024-09-21T23:17:24.118818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:29:21.964511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":36265277,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":12688,"num_read_kb":24820,"num_write":697,"num_write_kb":2932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"115'446","reported_seq":4060,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504450+0000","last_change":"2024-09-21T23:16:36.050207+0000","last_active":"2024-09-21T23:17:24.504450+0000","last_peered":"2024-09-21T23:17:24.504450+0000","last_clean":"2024-09-21T23:17:24.504450+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:17:24.504450+0000","last_undegraded":"2024-09-21T23:17:24.504450+0000","last_fullsized":"2024-09-21T23:17:24.504450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:21:57.161673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":15650023,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":5276,"num_read_kb":11757,"num_write":361,"num_write_kb":1633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"120'247","reported_seq":3049,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504263+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:17:24.504263+0000","last_peered":"2024-09-21T23:17:24.504263+0000","last_clean":"2024-09-21T23:17:24.504263+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:17:24.504263+0000","last_undegraded":"2024-09-21T23:17:24.504263+0000","last_fullsized":"2024-09-21T23:17:24.504263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:21.512989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021264,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":2517,"num_read_kb":14502,"num_write":225,"num_write_kb":1186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"115'644","reported_seq":7374,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.118962+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:17:24.118962+0000","last_peered":"2024-09-21T23:17:24.118962+0000","last_clean":"2024-09-21T23:17:24.118962+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:17:24.118962+0000","last_undegraded":"2024-09-21T23:17:24.118962+0000","last_fullsized":"2024-09-21T23:17:24.118962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:07:54.665875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":16158269,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":9758,"num_read_kb":19499,"num_write":598,"num_write_kb":1859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"121'412","reported_seq":3459,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.818146+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:17:24.818146+0000","last_peered":"2024-09-21T23:17:24.818146+0000","last_clean":"2024-09-21T23:17:24.818146+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:17:24.818146+0000","last_undegraded":"2024-09-21T23:17:24.818146+0000","last_fullsized":"2024-09-21T23:17:24.818146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:49:11.993843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":22625325,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":4292,"num_read_kb":8336,"num_write":340,"num_write_kb":2057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'335","reported_seq":3944,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504499+0000","last_change":"2024-09-21T23:16:36.052262+0000","last_active":"2024-09-21T23:17:24.504499+0000","last_peered":"2024-09-21T23:17:24.504499+0000","last_clean":"2024-09-21T23:17:24.504499+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:17:24.504499+0000","last_undegraded":"2024-09-21T23:17:24.504499+0000","last_fullsized":"2024-09-21T23:17:24.504499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:29:12.992703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7152434,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":4138,"num_read_kb":25418,"num_write":433,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"114'329","reported_seq":3248,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109749+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:17:24.109749+0000","last_peered":"2024-09-21T23:17:24.109749+0000","last_clean":"2024-09-21T23:17:24.109749+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:17:24.109749+0000","last_undegraded":"2024-09-21T23:17:24.109749+0000","last_fullsized":"2024-09-21T23:17:24.109749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:13:12.050563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":10085470,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":3880,"num_read_kb":11988,"num_write":322,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"117'360","reported_seq":4984,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.790030+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:17:24.790030+0000","last_peered":"2024-09-21T23:17:24.790030+0000","last_clean":"2024-09-21T23:17:24.790030+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:17:24.790030+0000","last_undegraded":"2024-09-21T23:17:24.790030+0000","last_fullsized":"2024-09-21T23:17:24.790030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:54:40.183577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":30326889,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":6436,"num_read_kb":19633,"num_write":328,"num_write_kb":1506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"118'590","reported_seq":3837,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109357+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:17:24.109357+0000","last_peered":"2024-09-21T23:17:24.109357+0000","last_clean":"2024-09-21T23:17:24.109357+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:17:24.109357+0000","last_undegraded":"2024-09-21T23:17:24.109357+0000","last_fullsized":"2024-09-21T23:17:24.109357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:33:45.358491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":17421417,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":4496,"num_read_kb":17102,"num_write":796,"num_write_kb":3825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"113'444","reported_seq":4638,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.504341+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:17:24.504341+0000","last_peered":"2024-09-21T23:17:24.504341+0000","last_clean":"2024-09-21T23:17:24.504341+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:17:24.504341+0000","last_undegraded":"2024-09-21T23:17:24.504341+0000","last_fullsized":"2024-09-21T23:17:24.504341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:48:36.890578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19368523,"num_objects":50,"num_object_clones":19,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":25,"num_read":5120,"num_read_kb":24578,"num_write":594,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"117'629","reported_seq":20883,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T23:17:21.046340+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:17:21.046340+0000","last_peered":"2024-09-21T23:17:21.046340+0000","last_clean":"2024-09-21T23:17:21.046340+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:17:21.046340+0000","last_undegraded":"2024-09-21T23:17:21.046340+0000","last_fullsized":"2024-09-21T23:17:21.046340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:18.868862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":37089895,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":23507,"num_read_kb":44206,"num_write":602,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"120'360","reported_seq":9258,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-21T23:17:24.109336+0000","last_change":"2024-09-21T23:16:36.050076+0000","last_active":"2024-09-21T23:17:24.109336+0000","last_peered":"2024-09-21T23:17:24.109336+0000","last_clean":"2024-09-21T23:17:24.109336+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:17:24.109336+0000","last_undegraded":"2024-09-21T23:17:24.109336+0000","last_fullsized":"2024-09-21T23:17:24.109336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:18:26.136287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269204,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":9040,"num_read_kb":21577,"num_write":401,"num_write_kb":1948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":840970051,"num_objects":1474,"num_object_clones":404,"num_object_copies":2948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1474,"num_whiteouts":658,"num_read":271674,"num_read_kb":876291,"num_write":17029,"num_write_kb":160868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2034,"num_bytes_recovered":22712186,"num_keys_recovered":282,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203010048,"data_stored":296640326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":15360,"ondisk_log_size":15360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106004,"kb_used_data":52600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94265836,"statfs":{"total":96636764160,"available":96528216064,"internally_reserved":0,"allocated":53862400,"data_stored":83031680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134148,"kb_used_data":76588,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94237692,"statfs":{"total":96636764160,"available":96499396608,"internally_reserved":0,"allocated":78426112,"data_stored":119567659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607589,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":120340,"kb_used_data":53048,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94251500,"statfs":{"total":96636764160,"available":96513536000,"internally_reserved":0,"allocated":54321152,"data_stored":70210533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005073,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50280,"kb_used_data":22104,"kb_used_omap":5,"kb_used_meta":28154,"kb_avail":94321560,"statfs":{"total":96636764160,"available":96585277440,"internally_reserved":0,"allocated":22634496,"data_stored":26777304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":28829989},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20819968,"data_stored":25774485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52301824,"data_stored":69034846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77004800,"data_stored":118982359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":52424704,"data_stored":82441420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:17:25.850 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:17:25.850 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:25.850 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T23:17:25.850 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:17:25.850 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:17:25.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T23:17:25.888 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:25.888 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:25.889 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:25.887+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:25.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:25.995 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:25.995+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:26.014 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "" 2024-09-21T23:17:26.015 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:26.070 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:26.070 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:26.080 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:26.079+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:26.087 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:26.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:17:26.181 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:26.176+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:26.227 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:26.227 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:26.240 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone33 from 0xe5b1400 (overlap 0x7a9a00) to 0x310a00 2024-09-21T23:17:26.241 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:26.286 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:26.284+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:26.382 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:26.380+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:26.407 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:26.407 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:26.422 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:26.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:26.510 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:26.508+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:26.583 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:26.583+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:26.586 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:26.586 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:26.596 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:26.597 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:26.600 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:26.614 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:26.614 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:26.630 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:26.630 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:26.649 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:26.667 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:26.684 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:26.683+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:26.685 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:26.686 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:26.769 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:26.769 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:26.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.6048.0:4 3.14 3:29797a19:::rbd_header.10f1c50d00c2:head [call rbd.get_stripe_unit_count in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123)", 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:26.768739+0000", 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00026507599999999999, 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000313178, 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.6048", 2024-09-21T23:17:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/56528173", 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 4 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:26.768739+0000", 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:26.768739+0000", 2024-09-21T23:17:26.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:26.768740+0000", 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3400000000000001e-06 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:26.768747+0000", 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6599999999999998e-06 2024-09-21T23:17:26.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:26.768766+0000", 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8703e-05 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:26.768777+0000", 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1610000000000001e-05 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:26.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:26.768902+0000", 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000124444 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:17:26.774 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:26.784 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:26.784 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:26.780+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:26.815 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-21T23:17:26.886 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:26.884+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:26.902 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:26.902 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:26.902 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:26.902 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:26.902 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:26.916 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:26.987 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:26.987+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 112.128829032, 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 112.12856204800001, 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.516233502999995, 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.516190804000004, 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.516152184000006, 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.516120311999998, 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.516087088000006, 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.516052013000007, 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.516007055000003, 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.515554988000005, 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.514840508999995, 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.514799439000001, 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.514342321000001, 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.514105642000004, 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.513690061000005, 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.157 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.513012400999997, 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:27.161 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.512521112000002, 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:27.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.512264213999998, 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:27.166 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:27.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:27.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.803073115, 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:17:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.800194414, 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:27.175 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:27.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.180 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.181 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.181 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:27.181 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:27.084+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:27.182 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:27.183 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:17:27.226 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:27.224+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:27.289 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:27.288+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:27.377 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:27.426 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:27.424+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:27.492 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:27.491+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:27.549 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:27.550 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:27.550 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:27.550 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:27.550 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:27.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:27.610 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:27.608+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:27.688 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:17:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.563544088, 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:17:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:17:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.559092930999995, 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:17:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:17:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.559065458000006, 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:17:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.559025285000004, 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:17:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:17:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558974516000006, 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:17:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558943932999995, 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558916026999995, 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:17:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:17:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558888807000002, 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:17:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558861887999996, 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:17:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:17:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558806125000004, 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:17:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558772951999998, 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:17:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:17:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558727732999998, 2024-09-21T23:17:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:17:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:17:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558688453000002, 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:17:27.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:17:27.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:17:27.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558657967000002, 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:17:27.753 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:17:27.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558628674000005, 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.758 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:27.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:17:27.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558586332000004, 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:17:27.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:27.766 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.558203137000007, 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:17:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:17:27.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.433355464, 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:27.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:27.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:17:27.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.408306505000001, 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:27.777 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:17:27.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.782 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.374873335, 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:17:27.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:17:27.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:27.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:27.787 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:27.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:27.789 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:27.787+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:27.889 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:27.884+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:27.925 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:27.925 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:27.925 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:27.925 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:27.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:27.990 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:27.988+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:28.058 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:28.058 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:28.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:28.119 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:28.119+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:28.191 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:28.191+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:28.202 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 86.040660982999995, 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.975406225, 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.578064092999995, 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.574696403999994, 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.570586824000003, 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.566793903000004, 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.506635095999997, 2024-09-21T23:17:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:28.234 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502878025000001, 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502851223999997, 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.239 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:28.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:28.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502825059000003, 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:28.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:28.247 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502796609000001, 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:28.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:28.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502705900999999, 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:28.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:28.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502674322000004, 2024-09-21T23:17:28.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502636632000005, 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:28.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:28.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.264 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502606241999999, 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.502562824999998, 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.269 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.5025282, 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:28.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:28.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:28.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.499274700000001, 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.499240118000003, 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:28.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.283495719000001, 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:28.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:28.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:28.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.291 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:28.292 INFO:teuthology.orchestra.run.smithi170.stdout:5301 read 0x593e00 thru 0x59e932 (0xab33 bytes) 2024-09-21T23:17:28.292 INFO:teuthology.orchestra.run.smithi170.stdout:5305 read 0xa95600 thru 0xa9de61 (0x8862 bytes) 2024-09-21T23:17:28.292 INFO:teuthology.orchestra.run.smithi170.stdout:5311 read 0xc10f600 thru 0xc111a3e (0x243f bytes) 2024-09-21T23:17:28.293 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:28.291+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:28.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:28.331 INFO:teuthology.orchestra.run.smithi170.stdout:5312 trunc from 0xe5b1400 to 0x2ebe00 2024-09-21T23:17:28.393 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:28.391+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:28.429 INFO:teuthology.orchestra.run.smithi170.stdout:5313 write 0x7457e00 thru 0x745a8c4 (0x2ac5 bytes) 2024-09-21T23:17:28.431 INFO:teuthology.orchestra.run.smithi170.stdout:5315 read 0x6b39000 thru 0x6b3e319 (0x531a bytes) 2024-09-21T23:17:28.471 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:28.471 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:28.471 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:28.472 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:28.477 INFO:teuthology.orchestra.run.smithi170.stdout:5317 write 0xe1fc600 thru 0xe20b7d1 (0xf1d2 bytes) 2024-09-21T23:17:28.479 INFO:teuthology.orchestra.run.smithi170.stdout:5321 flatten 2024-09-21T23:17:28.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:28.494 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:28.492+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:28.512 INFO:teuthology.orchestra.run.smithi170.stdout:5322 punch from 0xba28c00 to 0xba38600, (0xfa00 bytes) 2024-09-21T23:17:28.549 INFO:teuthology.orchestra.run.smithi170.stdout:5325 trunc from 0xe20b800 to 0x5bde00 2024-09-21T23:17:28.589 INFO:teuthology.orchestra.run.smithi170.stdout:5326 read 0xce000 thru 0xd9283 (0xb284 bytes) 2024-09-21T23:17:28.595 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:28.592+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:28.627 INFO:teuthology.orchestra.run.smithi170.stdout:5327 write 0x41c9e00 thru 0x41d1a25 (0x7c26 bytes) 2024-09-21T23:17:28.629 INFO:teuthology.orchestra.run.smithi170.stdout:5328 read 0x227e200 thru 0x2286370 (0x8171 bytes) 2024-09-21T23:17:28.629 INFO:teuthology.orchestra.run.smithi170.stdout:5329 read 0x2159200 thru 0x215b3af (0x21b0 bytes) 2024-09-21T23:17:28.664 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:28.664 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:28.677 INFO:teuthology.orchestra.run.smithi170.stdout:5330 write 0xa963e00 thru 0xa969084 (0x5285 bytes) 2024-09-21T23:17:28.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:28.686 INFO:teuthology.orchestra.run.smithi170.stdout:5333 trunc from 0xa969200 to 0x71dac00 2024-09-21T23:17:28.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:28.727 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:28.727+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:28.742 INFO:teuthology.orchestra.run.smithi170.stdout:5335 trunc from 0x71dac00 to 0x1810e00 2024-09-21T23:17:28.797 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:28.795+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:28.815 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.815 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054739694999995, 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:28.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:28.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054694437999999, 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:28.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:28.826 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054644816000007, 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054615120999998, 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054582193000002, 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054547897000006, 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054523391000004, 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054495400999997, 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054472372999996, 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054450863, 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054415321999997, 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054387684999995, 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054352520999998, 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054317490000003, 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054282104999999, 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054241923000006, 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054204795000004, 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054164720000003, 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054140176000004, 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.054102391000001, 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:28.915 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:28.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:28.917 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:28.918 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:28.918 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:28.915+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:28.920 INFO:teuthology.orchestra.run.smithi170.stdout:5337 write 0xd175200 thru 0xd183397 (0xe198 bytes) 2024-09-21T23:17:28.920 INFO:teuthology.orchestra.run.smithi170.stdout:5338 read 0x796e200 thru 0x797b936 (0xd737 bytes) 2024-09-21T23:17:28.920 INFO:teuthology.orchestra.run.smithi170.stdout:5340 write 0xcf13e00 thru 0xcf1c8ef (0x8af0 bytes) 2024-09-21T23:17:28.920 INFO:teuthology.orchestra.run.smithi170.stdout:5341 read 0x73a4800 thru 0x73ac9b8 (0x81b9 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5343 read 0xc3fb400 thru 0xc3fd97f (0x2580 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5345 punch from 0x80c1200 to 0x80c9800, (0x8600 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5346 write 0x334b200 thru 0x334c9f9 (0x17fa bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5347 write 0x1a6e200 thru 0x1a75a33 (0x7834 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5348 read 0x5a0d600 thru 0x5a0d945 (0x346 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5351 write 0x3e8d600 thru 0x3e9b93a (0xe33b bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5355 read 0x5e33c00 thru 0x5e3c3e7 (0x87e8 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5357 write 0x57c0400 thru 0x57c89cd (0x85ce bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5358 write 0x3053c00 thru 0x3058a83 (0x4e84 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5359 read 0x1d45e00 thru 0x1d50021 (0xa222 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5362 write 0xcbbc00 thru 0xcca0e1 (0xe4e2 bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5364 read 0x4e13200 thru 0x4e133cd (0x1ce bytes) 2024-09-21T23:17:28.921 INFO:teuthology.orchestra.run.smithi170.stdout:5366 punch from 0x974d000 to 0x9752e00, (0x5e00 bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5367 read 0x4b47600 thru 0x4b497fd (0x21fe bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5368 write 0xc01a200 thru 0xc027b09 (0xd90a bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5370 punch from 0x748ca00 to 0x7490200, (0x3800 bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5371 read 0xaed2a00 thru 0xaeda5ec (0x7bed bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5373 trunc from 0xd183400 to 0xd2b9a00 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5375 punch from 0x18aca00 to 0x18b5600, (0x8c00 bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5376 read 0x8419600 thru 0x84232da (0x9cdb bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5377 write 0x179600 thru 0x17ba0a (0x240b bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5378 write 0x1f32600 thru 0x1f37f66 (0x5967 bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5379 read 0xaec5400 thru 0xaed36ad (0xe2ae bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5380 read 0x241b400 thru 0x2426d63 (0xb964 bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5381 write 0x3914c00 thru 0x391cd4f (0x8150 bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5383 write 0x8c7e00 thru 0x8d7913 (0xfb14 bytes) 2024-09-21T23:17:28.922 INFO:teuthology.orchestra.run.smithi170.stdout:5384 read 0x2a4b200 thru 0x2a4d703 (0x2504 bytes) 2024-09-21T23:17:28.923 INFO:teuthology.orchestra.run.smithi170.stdout:5385 trunc from 0xd2b9a00 to 0xb062a00 2024-09-21T23:17:28.937 INFO:teuthology.orchestra.run.smithi170.stdout:5386 write 0x86ea600 thru 0x86eae30 (0x831 bytes) 2024-09-21T23:17:28.939 INFO:teuthology.orchestra.run.smithi170.stdout:5387 read 0x1773200 thru 0x177d686 (0xa487 bytes) 2024-09-21T23:17:28.939 INFO:teuthology.orchestra.run.smithi170.stdout:5388 read 0x6e08000 thru 0x6e1597c (0xd97d bytes) 2024-09-21T23:17:28.940 INFO:teuthology.orchestra.run.smithi170.stdout:5392 write 0x18ada00 thru 0x18b4667 (0x6c68 bytes) 2024-09-21T23:17:28.942 INFO:teuthology.orchestra.run.smithi170.stdout:5393 trunc from 0xb062a00 to 0xb612000 2024-09-21T23:17:28.970 INFO:teuthology.orchestra.run.smithi170.stdout:5398 punch from 0x3df5c00 to 0x3df8000, (0x2400 bytes) 2024-09-21T23:17:28.971 INFO:teuthology.orchestra.run.smithi170.stdout:5399 clone 35 order 21 su 524288 sc 7 2024-09-21T23:17:29.017 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:29.012+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:29.087 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:29.087 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:29.106 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:29.112 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:29.112 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:29.118 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:29.116+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:29.132 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:29.133 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:29.154 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:29.155 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:29.181 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:29.182 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:29.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:17:29.208 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:29.209 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:29.218 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:29.216+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:29.228 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:29.228 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:29.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:29.339 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:29.339+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:29.418 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:29.418 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:29.420 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:29.416+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:29.433 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:29.521 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:29.516+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:29.568 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:29.569 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:29.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:29.626 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:29.624+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:29.723 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:29.720+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:29.756 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:29.756 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:29.770 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:29.823 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:29.823+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:29.878 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-21T23:17:29.924 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:29.923+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:29.940 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:29.940 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:29.955 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:29.995 INFO:teuthology.orchestra.run.smithi170.stdout:5400 punch from 0x1b2f400 to 0x1b3ba00, (0xc600 bytes) 2024-09-21T23:17:30.019 INFO:teuthology.orchestra.run.smithi170.stdout:5406 trunc from 0xb612000 to 0xc4e0a00 2024-09-21T23:17:30.025 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:30.023+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:30.083 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:30.083 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:30.097 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:17:30.116 INFO:teuthology.orchestra.run.smithi170.stdout:5409 trunc from 0xc4e0a00 to 0x4435a00 2024-09-21T23:17:30.125 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:30.123+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:30.147 INFO:teuthology.orchestra.run.smithi170.stdout:5413 write 0x2efb600 thru 0x2f08285 (0xcc86 bytes) 2024-09-21T23:17:30.149 INFO:teuthology.orchestra.run.smithi170.stdout:5414 punch from 0x425e800 to 0x4269600, (0xae00 bytes) 2024-09-21T23:17:30.150 INFO:teuthology.orchestra.run.smithi170.stdout:5415 read 0xf3fa00 thru 0xf4f3b2 (0xf9b3 bytes) 2024-09-21T23:17:30.173 INFO:teuthology.orchestra.run.smithi170.stdout:5416 trunc from 0x4435a00 to 0xc786e00 2024-09-21T23:17:30.191 INFO:teuthology.orchestra.run.smithi170.stdout:5417 write 0x7ef4e00 thru 0x7ef7094 (0x2295 bytes) 2024-09-21T23:17:30.194 INFO:teuthology.orchestra.run.smithi170.stdout:5420 read 0x8f18200 thru 0x8f1f8c8 (0x76c9 bytes) 2024-09-21T23:17:30.207 INFO:teuthology.orchestra.run.smithi170.stdout:5422 trunc from 0xc786e00 to 0x7194a00 2024-09-21T23:17:30.226 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:30.224+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:30.228 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:30.228 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:30.244 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:30.245 INFO:teuthology.orchestra.run.smithi170.stdout:5424 write 0x61c6800 thru 0x61d6770 (0xff71 bytes) 2024-09-21T23:17:30.247 INFO:teuthology.orchestra.run.smithi170.stdout:5425 write 0x6e55000 thru 0x6e6421e (0xf21f bytes) 2024-09-21T23:17:30.247 INFO:teuthology.orchestra.run.smithi170.stdout:5428 punch from 0x591b200 to 0x5927200, (0xc000 bytes) 2024-09-21T23:17:30.248 INFO:teuthology.orchestra.run.smithi170.stdout:5430 write 0x3649600 thru 0x365239b (0x8d9c bytes) 2024-09-21T23:17:30.249 INFO:teuthology.orchestra.run.smithi170.stdout:5433 write 0x4baac00 thru 0x4babec1 (0x12c2 bytes) 2024-09-21T23:17:30.253 INFO:teuthology.orchestra.run.smithi170.stdout:5434 punch from 0x1e6b400 to 0x1e77e00, (0xca00 bytes) 2024-09-21T23:17:30.253 INFO:teuthology.orchestra.run.smithi170.stdout:5435 write 0x2e37600 thru 0x2e3e684 (0x7085 bytes) 2024-09-21T23:17:30.254 INFO:teuthology.orchestra.run.smithi170.stdout:5437 punch from 0x6915e00 to 0x6918600, (0x2800 bytes) 2024-09-21T23:17:30.266 INFO:teuthology.orchestra.run.smithi170.stdout:5439 trunc from 0x7194a00 to 0xbc37000 2024-09-21T23:17:30.288 INFO:teuthology.orchestra.run.smithi170.stdout:5440 write 0x9c20200 thru 0x9c24494 (0x4295 bytes) 2024-09-21T23:17:30.290 INFO:teuthology.orchestra.run.smithi170.stdout:5441 read 0x9c7f800 thru 0x9c88aca (0x92cb bytes) 2024-09-21T23:17:30.290 INFO:teuthology.orchestra.run.smithi170.stdout:5442 write 0x8978a00 thru 0x8984d5a (0xc35b bytes) 2024-09-21T23:17:30.291 INFO:teuthology.orchestra.run.smithi170.stdout:5443 read 0xb047200 thru 0xb04ebd2 (0x79d3 bytes) 2024-09-21T23:17:30.291 INFO:teuthology.orchestra.run.smithi170.stdout:5444 write 0x3bf4e00 thru 0x3bfed9c (0x9f9d bytes) 2024-09-21T23:17:30.291 INFO:teuthology.orchestra.run.smithi170.stdout:5445 write 0x9c78000 thru 0x9c7beb6 (0x3eb7 bytes) 2024-09-21T23:17:30.291 INFO:teuthology.orchestra.run.smithi170.stdout:5446 read 0x4ddb000 thru 0x4de8f8f (0xdf90 bytes) 2024-09-21T23:17:30.331 INFO:teuthology.orchestra.run.smithi170.stdout:5447 write 0xd1a6c00 thru 0xd1b0740 (0x9b41 bytes) 2024-09-21T23:17:30.333 INFO:teuthology.orchestra.run.smithi170.stdout:5450 read 0xd0a9800 thru 0xd0addc9 (0x45ca bytes) 2024-09-21T23:17:30.333 INFO:teuthology.orchestra.run.smithi170.stdout:5451 clone 36 order 23 su 524288 sc 2 2024-09-21T23:17:30.338 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:30.336+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:30.411 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:30.411 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:30.425 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:30.428 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:30.427+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:30.529 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:30.524+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:30.591 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:30.592 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:30.606 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:30.631 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:30.631+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:30.731 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:30.728+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:30.776 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:30.776 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:30.790 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:30.833 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:30.831+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:30.933 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:30.931+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:30.953 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:30.953 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:30.967 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:31.031 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:31.031 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:31.031 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T23:17:31.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:17:31.034 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:31.032+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:31.102 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:31.103 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:31.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:31.134 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:31.132+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:31.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:31.252 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:31.252 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:31.252 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:31.251+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:31.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:31.270 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone35 from 0xd1b0800 (overlap 0x4435a00) to 0xda4400 2024-09-21T23:17:31.355 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:31.355+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 127, 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "309803bf-3365-47ac-8bf6-365125beceb1", 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-21T23:14:32.028556+0000", 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-21T23:17:31.112020+0000", 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-21T23:16:02.149456+0000", 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-21T23:16:51.115738+0000", 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5804032, 2024-09-21T23:17:31.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 6, 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-21T23:17:31.370 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 6, 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "jewel", 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "squid", 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:44.926374+0000", 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:17:31.371 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-21T23:17:31.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "15", 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:17:31.373 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:17:31.374 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:17:31.375 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.376 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 6.059999942779541, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 6.059999942779541, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-21T23:17:31.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:49.598603+0000", 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:17:31.378 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:17:31.379 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "19", 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 2, 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 19, 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:17:31.380 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:17:31.381 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:17:31.382 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2.9900000095367432, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2.9900000095367432, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.67000001668930054, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:17:31.383 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:59.367341+0000", 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:17:31.384 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-21T23:17:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "127", 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:17:31.386 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 38, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 127, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:17:31.387 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:17:31.388 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.690000057220459, 2024-09-21T23:17:31.389 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.690000057220459, 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.67000001668930054, 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:17:31.390 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "a24df615-4443-4f27-888f-d1e4e7affbb3", 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 12, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 29, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 61, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 31, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:17:31.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1028371170", 2024-09-21T23:17:31.395 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1028371170", 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1028371170", 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1028371170", 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d6765c2a-98b2-4a2b-8d8c-f3963575eb64", 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:17:31.396 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/2712042978", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/2712042978", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2712042978", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2712042978", 2024-09-21T23:17:31.401 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "9753272a-58a9-4366-87d7-550c49134023", 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:17:31.402 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 74, 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-21T23:17:31.405 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-21T23:17:31.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/1166517858", 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/1166517858", 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/1166517858", 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/1166517858", 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.407 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "24d987f9-f2cd-4ccd-9dc8-fea64467d92d", 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 21, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6809", 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6811", 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6814", 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6816", 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.410 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6822", 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6823", 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6818", 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6821", 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6811/1039392120", 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6816/1039392120", 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6823/1039392120", 2024-09-21T23:17:31.412 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6821/1039392120", 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "c1c30a49-7f26-4264-a1a9-160dc552953d", 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:17:31.413 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6808", 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6810", 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6812", 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6813", 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6819", 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6820", 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6815", 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6817", 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6810/2764095640", 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6813/2764095640", 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6820/2764095640", 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6817/2764095640", 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f24e9341-be4e-4be2-b180-7ebd3f705e9b", 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 99, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6800", 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6801", 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6802", 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6803", 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6806", 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6807", 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:17:31.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6804", 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6805", 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6801/1732973431", 2024-09-21T23:17:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6803/1732973431", 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6807/1732973431", 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6805/1732973431", 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:17:31.424 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.468749+0000", 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 31 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.044837+0000", 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:17:31.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 8196, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.310970+0000", 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:17:31.426 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.241402+0000", 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.877818+0000", 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.434052+0000", 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-21T23:17:31.428 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-21T23:17:31.429 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.430 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:31.431 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:31.431 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:31.432 INFO:tasks.thrashosds.thrasher:j is {'epoch': 127, 'fsid': '309803bf-3365-47ac-8bf6-365125beceb1', 'created': '2024-09-21T23:14:32.028556+0000', 'modified': '2024-09-21T23:17:31.112020+0000', 'last_up_change': '2024-09-21T23:16:02.149456+0000', 'last_in_change': '2024-09-21T23:16:51.115738+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:14:44.926374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:14:49.598603+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T23:14:59.367341+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 38, 'snap_epoch': 127, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a24df615-4443-4f27-888f-d1e4e7affbb3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 12, 'last_clean_end': 29, 'up_from': 61, 'up_thru': 87, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1028371170}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1028371170}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1028371170}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1028371170}]}, 'public_addr': '172.21.15.53:6817/1028371170', 'cluster_addr': '172.21.15.53:6819/1028371170', 'heartbeat_back_addr': '172.21.15.53:6823/1028371170', 'heartbeat_front_addr': '172.21.15.53:6821/1028371170', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd6765c2a-98b2-4a2b-8d8c-f3963575eb64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2712042978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2712042978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2712042978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2712042978}]}, 'public_addr': '172.21.15.53:6809/2712042978', 'cluster_addr': '172.21.15.53:6811/2712042978', 'heartbeat_back_addr': '172.21.15.53:6815/2712042978', 'heartbeat_front_addr': '172.21.15.53:6813/2712042978', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9753272a-58a9-4366-87d7-550c49134023', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 1166517858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 1166517858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 1166517858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 1166517858}]}, 'public_addr': '172.21.15.53:6801/1166517858', 'cluster_addr': '172.21.15.53:6803/1166517858', 'heartbeat_back_addr': '172.21.15.53:6807/1166517858', 'heartbeat_front_addr': '172.21.15.53:6805/1166517858', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '24d987f9-f2cd-4ccd-9dc8-fea64467d92d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6809', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 1039392120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6816', 'nonce': 1039392120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6823', 'nonce': 1039392120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 1039392120}]}, 'public_addr': '172.21.15.144:6811/1039392120', 'cluster_addr': '172.21.15.144:6816/1039392120', 'heartbeat_back_addr': '172.21.15.144:6823/1039392120', 'heartbeat_front_addr': '172.21.15.144:6821/1039392120', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'c1c30a49-7f26-4264-a1a9-160dc552953d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6810', 'nonce': 2764095640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 2764095640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2764095640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6815', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2764095640}]}, 'public_addr': '172.21.15.144:6810/2764095640', 'cluster_addr': '172.21.15.144:6813/2764095640', 'heartbeat_back_addr': '172.21.15.144:6820/2764095640', 'heartbeat_front_addr': '172.21.15.144:6817/2764095640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f24e9341-be4e-4be2-b180-7ebd3f705e9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 1732973431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 1732973431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 1732973431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 1732973431}]}, 'public_addr': '172.21.15.144:6801/1732973431', 'cluster_addr': '172.21.15.144:6803/1732973431', 'heartbeat_back_addr': '172.21.15.144:6807/1732973431', 'heartbeat_front_addr': '172.21.15.144:6805/1732973431', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.468749+0000', 'dead_epoch': 31}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.044837+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 8196, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.310970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.241402+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.877818+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.434052+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:17:31.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:17:31.433 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:31.433 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:31.433 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:31.433 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.276273631999999, 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:31.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:31.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:31.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.211018874000004, 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.813676741999998, 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:31.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:31.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.810309052999997, 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:31.450 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:31.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:31.454 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.806199473000007, 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:31.458 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.802406551999994, 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:31.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:31.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8002 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4012032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.742247745, 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083087402000000005, 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8002 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.695410+0000", 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698868+0000", 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034576899999999998 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698879+0000", 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1139e-05 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698892+0000", 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.698900+0000", 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5010000000000004e-06 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778085+0000", 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079184289000000005 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778260+0000", 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000175532 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.778497+0000", 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023726199999999999 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.466 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738490674000005, 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:31.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738463873000001, 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.471 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738437708000006, 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738409258000004, 2024-09-21T23:17:31.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738318550000002, 2024-09-21T23:17:31.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:31.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.487 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738286970999994, 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:31.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:31.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738249280999995, 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738218891000002, 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:31.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738175474000002, 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:31.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.738140849000004, 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.505 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:31.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:31.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.734887349000005, 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:31.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:31.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:31.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:31.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.734852767000007, 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:31.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:31.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:31.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:31.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.519108368000005, 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:31.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:31.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:31.523 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:31.523 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:31.435+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:31.524 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:31.524 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:31.525 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:31.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:31.525 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:31.526 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:31.526 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:31.567 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:31.567+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:31.637 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:31.635+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:31.689 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:31.689 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:31.689 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:31.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:31.737 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:31.736+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:17:31.834 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:17:31.835 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-21T23:17:31.838 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:31.835+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:31.853 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-21T23:17:31.116527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":881547371,"num_objects":1507,"num_object_clones":405,"num_object_copies":3014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1507,"num_whiteouts":658,"num_read":277535,"num_read_kb":899015,"num_write":17513,"num_write_kb":168384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":23892960,"num_keys_recovered":282,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15786,"ondisk_log_size":15786,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":422380,"kb_used_data":211768,"kb_used_omap":10,"kb_used_meta":210485,"kb_avail":377064980,"statfs":{"total":386547056640,"available":386114539520,"internally_reserved":0,"allocated":216850432,"data_stored":308969391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":215537268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16534757,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":3988,"num_read_kb":19327,"num_write":265,"num_write_kb":4436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.230113"},"pg_stats":[{"pgid":"3.1f","version":"123'450","reported_seq":12453,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.141107+0000","last_change":"2024-09-21T23:16:36.051323+0000","last_active":"2024-09-21T23:17:26.141107+0000","last_peered":"2024-09-21T23:17:26.141107+0000","last_clean":"2024-09-21T23:17:26.141107+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:17:26.141107+0000","last_undegraded":"2024-09-21T23:17:26.141107+0000","last_fullsized":"2024-09-21T23:17:26.141107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:02.126986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013689080000000001,"stat_sum":{"num_bytes":47580249,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":12502,"num_read_kb":20092,"num_write":542,"num_write_kb":3403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"124'392","reported_seq":4844,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120975+0000","last_change":"2024-09-21T23:16:36.049621+0000","last_active":"2024-09-21T23:17:29.120975+0000","last_peered":"2024-09-21T23:17:29.120975+0000","last_clean":"2024-09-21T23:17:29.120975+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:17:29.120975+0000","last_undegraded":"2024-09-21T23:17:29.120975+0000","last_fullsized":"2024-09-21T23:17:29.120975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:50.032592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042130999999999999,"stat_sum":{"num_bytes":24049731,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":5018,"num_read_kb":25750,"num_write":351,"num_write_kb":6229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"124'374","reported_seq":7064,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.146396+0000","last_change":"2024-09-21T23:15:03.321054+0000","last_active":"2024-09-21T23:17:29.146396+0000","last_peered":"2024-09-21T23:17:29.146396+0000","last_clean":"2024-09-21T23:17:29.146396+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:17:29.146396+0000","last_undegraded":"2024-09-21T23:17:29.146396+0000","last_fullsized":"2024-09-21T23:17:29.146396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:31:40.441291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019165429999999999,"stat_sum":{"num_bytes":41979428,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":7479,"num_read_kb":36644,"num_write":306,"num_write_kb":17191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"125'612","reported_seq":10556,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.828142+0000","last_change":"2024-09-21T23:15:03.321202+0000","last_active":"2024-09-21T23:17:29.828142+0000","last_peered":"2024-09-21T23:17:29.828142+0000","last_clean":"2024-09-21T23:17:29.828142+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:17:29.828142+0000","last_undegraded":"2024-09-21T23:17:29.828142+0000","last_fullsized":"2024-09-21T23:17:29.828142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:03:37.219285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001962283,"stat_sum":{"num_bytes":34172496,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":14255,"num_read_kb":51738,"num_write":448,"num_write_kb":14154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"122'346","reported_seq":4354,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089405+0000","last_change":"2024-09-21T23:16:06.220156+0000","last_active":"2024-09-21T23:17:26.089405+0000","last_peered":"2024-09-21T23:17:26.089405+0000","last_clean":"2024-09-21T23:17:26.089405+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:17:26.089405+0000","last_undegraded":"2024-09-21T23:17:26.089405+0000","last_fullsized":"2024-09-21T23:17:26.089405+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:09:28.653471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015944990000000001,"stat_sum":{"num_bytes":12618872,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":4875,"num_read_kb":20010,"num_write":357,"num_write_kb":1365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"125'451","reported_seq":9051,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.845037+0000","last_change":"2024-09-21T23:16:18.779498+0000","last_active":"2024-09-21T23:17:29.845037+0000","last_peered":"2024-09-21T23:17:29.845037+0000","last_clean":"2024-09-21T23:17:29.845037+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:17:29.845037+0000","last_undegraded":"2024-09-21T23:17:29.845037+0000","last_fullsized":"2024-09-21T23:17:29.845037+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:06:14.147594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002136841,"stat_sum":{"num_bytes":11755124,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":10349,"num_read_kb":24546,"num_write":390,"num_write_kb":1332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"122'308","reported_seq":4377,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120594+0000","last_change":"2024-09-21T23:16:17.764987+0000","last_active":"2024-09-21T23:17:29.120594+0000","last_peered":"2024-09-21T23:17:29.120594+0000","last_clean":"2024-09-21T23:17:29.120594+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:17:29.120594+0000","last_undegraded":"2024-09-21T23:17:29.120594+0000","last_fullsized":"2024-09-21T23:17:29.120594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:23.762444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241177,"stat_sum":{"num_bytes":58400816,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":4186,"num_read_kb":61491,"num_write":253,"num_write_kb":30841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"114'520","reported_seq":17472,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089815+0000","last_change":"2024-09-21T23:16:36.046053+0000","last_active":"2024-09-21T23:17:26.089815+0000","last_peered":"2024-09-21T23:17:26.089815+0000","last_clean":"2024-09-21T23:17:26.089815+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:17:26.089815+0000","last_undegraded":"2024-09-21T23:17:26.089815+0000","last_fullsized":"2024-09-21T23:17:26.089815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:17.328694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729059,"stat_sum":{"num_bytes":55256695,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":15979,"num_read_kb":49681,"num_write":645,"num_write_kb":11238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"116'326","reported_seq":13019,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089677+0000","last_change":"2024-09-21T23:16:06.214612+0000","last_active":"2024-09-21T23:17:26.089677+0000","last_peered":"2024-09-21T23:17:26.089677+0000","last_clean":"2024-09-21T23:17:26.089677+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:17:26.089677+0000","last_undegraded":"2024-09-21T23:17:26.089677+0000","last_fullsized":"2024-09-21T23:17:26.089677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:33.677212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013405299999999999,"stat_sum":{"num_bytes":25262205,"num_objects":54,"num_object_clones":18,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":13051,"num_read_kb":33576,"num_write":279,"num_write_kb":2297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"114'472","reported_seq":5329,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.147182+0000","last_change":"2024-09-21T23:16:18.769499+0000","last_active":"2024-09-21T23:17:29.147182+0000","last_peered":"2024-09-21T23:17:29.147182+0000","last_clean":"2024-09-21T23:17:29.147182+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:17:29.147182+0000","last_undegraded":"2024-09-21T23:17:29.147182+0000","last_fullsized":"2024-09-21T23:17:29.147182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:31.493715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025421380000000002,"stat_sum":{"num_bytes":45419208,"num_objects":82,"num_object_clones":30,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":5313,"num_read_kb":21318,"num_write":539,"num_write_kb":3199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"118'294","reported_seq":3652,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089547+0000","last_change":"2024-09-21T23:16:06.214013+0000","last_active":"2024-09-21T23:17:26.089547+0000","last_peered":"2024-09-21T23:17:26.089547+0000","last_clean":"2024-09-21T23:17:26.089547+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:17:26.089547+0000","last_undegraded":"2024-09-21T23:17:26.089547+0000","last_fullsized":"2024-09-21T23:17:26.089547+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:40:23.203448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23419427,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":3616,"num_read_kb":26735,"num_write":341,"num_write_kb":3576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"124'782","reported_seq":7202,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.835360+0000","last_change":"2024-09-21T23:15:06.354455+0000","last_active":"2024-09-21T23:17:29.835360+0000","last_peered":"2024-09-21T23:17:29.835360+0000","last_clean":"2024-09-21T23:17:29.835360+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:17:29.835360+0000","last_undegraded":"2024-09-21T23:17:29.835360+0000","last_fullsized":"2024-09-21T23:17:29.835360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:43:51.070317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31251653,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":17,"num_read":10427,"num_read_kb":34942,"num_write":971,"num_write_kb":12440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":215,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.107372+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:17:29.107372+0000","last_peered":"2024-09-21T23:17:29.107372+0000","last_clean":"2024-09-21T23:17:29.107372+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:17:29.107372+0000","last_undegraded":"2024-09-21T23:17:29.107372+0000","last_fullsized":"2024-09-21T23:17:29.107372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"117'655","reported_seq":19635,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089596+0000","last_change":"2024-09-21T23:16:18.775397+0000","last_active":"2024-09-21T23:17:26.089596+0000","last_peered":"2024-09-21T23:17:26.089596+0000","last_clean":"2024-09-21T23:17:26.089596+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:17:26.089596+0000","last_undegraded":"2024-09-21T23:17:26.089596+0000","last_fullsized":"2024-09-21T23:17:26.089596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:20.062812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091740300000000001,"stat_sum":{"num_bytes":45253731,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":20617,"num_read_kb":32866,"num_write":975,"num_write_kb":4463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120975+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:17:29.120975+0000","last_peered":"2024-09-21T23:17:29.120975+0000","last_clean":"2024-09-21T23:17:29.120975+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:17:29.120975+0000","last_undegraded":"2024-09-21T23:17:29.120975+0000","last_fullsized":"2024-09-21T23:17:29.120975+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'1576","reported_seq":7476,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.717297+0000","last_change":"2024-09-21T23:16:36.052892+0000","last_active":"2024-09-21T23:17:29.717297+0000","last_peered":"2024-09-21T23:17:29.717297+0000","last_clean":"2024-09-21T23:17:29.717297+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:17:29.717297+0000","last_undegraded":"2024-09-21T23:17:29.717297+0000","last_fullsized":"2024-09-21T23:17:29.717297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1576,"log_dups_size":0,"ondisk_log_size":1576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:02.303071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062810999999999997,"stat_sum":{"num_bytes":13298144,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":12629,"num_read_kb":39785,"num_write":2739,"num_write_kb":7110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":144,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089501+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:17:26.089501+0000","last_peered":"2024-09-21T23:17:26.089501+0000","last_clean":"2024-09-21T23:17:26.089501+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:17:26.089501+0000","last_undegraded":"2024-09-21T23:17:26.089501+0000","last_fullsized":"2024-09-21T23:17:26.089501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"122'353","reported_seq":1699,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120705+0000","last_change":"2024-09-21T23:16:36.050246+0000","last_active":"2024-09-21T23:17:29.120705+0000","last_peered":"2024-09-21T23:17:29.120705+0000","last_clean":"2024-09-21T23:17:29.120705+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:17:29.120705+0000","last_undegraded":"2024-09-21T23:17:29.120705+0000","last_fullsized":"2024-09-21T23:17:29.120705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:08.134674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018003979999999999,"stat_sum":{"num_bytes":9744158,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":2046,"num_read_kb":3855,"num_write":382,"num_write_kb":1614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120684+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:17:29.120684+0000","last_peered":"2024-09-21T23:17:29.120684+0000","last_clean":"2024-09-21T23:17:29.120684+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:17:29.120684+0000","last_undegraded":"2024-09-21T23:17:29.120684+0000","last_fullsized":"2024-09-21T23:17:29.120684+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"124'439","reported_seq":9545,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.147021+0000","last_change":"2024-09-21T23:16:54.784043+0000","last_active":"2024-09-21T23:17:29.147021+0000","last_peered":"2024-09-21T23:17:29.147021+0000","last_clean":"2024-09-21T23:17:29.147021+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:17:29.147021+0000","last_undegraded":"2024-09-21T23:17:29.147021+0000","last_fullsized":"2024-09-21T23:17:29.147021+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:13:11.082940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023113140000000001,"stat_sum":{"num_bytes":48139027,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":19,"num_read":10452,"num_read_kb":68452,"num_write":528,"num_write_kb":4035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"115'527","reported_seq":5818,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.107385+0000","last_change":"2024-09-21T23:16:06.215552+0000","last_active":"2024-09-21T23:17:29.107385+0000","last_peered":"2024-09-21T23:17:29.107385+0000","last_clean":"2024-09-21T23:17:29.107385+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:17:29.107385+0000","last_undegraded":"2024-09-21T23:17:29.107385+0000","last_fullsized":"2024-09-21T23:17:29.107385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:44.453257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388770000000001,"stat_sum":{"num_bytes":15034029,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":7460,"num_read_kb":22133,"num_write":519,"num_write_kb":2170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":254,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.146019+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:17:29.146019+0000","last_peered":"2024-09-21T23:17:29.146019+0000","last_clean":"2024-09-21T23:17:29.146019+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:17:29.146019+0000","last_undegraded":"2024-09-21T23:17:29.146019+0000","last_fullsized":"2024-09-21T23:17:29.146019+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"124'286","reported_seq":5676,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120707+0000","last_change":"2024-09-21T23:16:36.051625+0000","last_active":"2024-09-21T23:17:29.120707+0000","last_peered":"2024-09-21T23:17:29.120707+0000","last_clean":"2024-09-21T23:17:29.120707+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:17:29.120707+0000","last_undegraded":"2024-09-21T23:17:29.120707+0000","last_fullsized":"2024-09-21T23:17:29.120707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:51:07.678745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258861,"stat_sum":{"num_bytes":24109684,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":5345,"num_read_kb":21781,"num_write":242,"num_write_kb":1338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":219,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120410+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:17:29.120410+0000","last_peered":"2024-09-21T23:17:29.120410+0000","last_clean":"2024-09-21T23:17:29.120410+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:17:29.120410+0000","last_undegraded":"2024-09-21T23:17:29.120410+0000","last_fullsized":"2024-09-21T23:17:29.120410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"122'337","reported_seq":4252,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089707+0000","last_change":"2024-09-21T23:16:06.219603+0000","last_active":"2024-09-21T23:17:26.089707+0000","last_peered":"2024-09-21T23:17:26.089707+0000","last_clean":"2024-09-21T23:17:26.089707+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:17:26.089707+0000","last_undegraded":"2024-09-21T23:17:26.089707+0000","last_fullsized":"2024-09-21T23:17:26.089707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:30:52.851745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061373600000000003,"stat_sum":{"num_bytes":36428399,"num_objects":52,"num_object_clones":17,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":3798,"num_read_kb":15137,"num_write":327,"num_write_kb":5688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120459+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:17:29.120459+0000","last_peered":"2024-09-21T23:17:29.120459+0000","last_clean":"2024-09-21T23:17:29.120459+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:17:29.120459+0000","last_undegraded":"2024-09-21T23:17:29.120459+0000","last_fullsized":"2024-09-21T23:17:29.120459+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"124'434","reported_seq":13161,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.710843+0000","last_change":"2024-09-21T23:16:17.769301+0000","last_active":"2024-09-21T23:17:29.710843+0000","last_peered":"2024-09-21T23:17:29.710843+0000","last_clean":"2024-09-21T23:17:29.710843+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:17:29.710843+0000","last_undegraded":"2024-09-21T23:17:29.710843+0000","last_fullsized":"2024-09-21T23:17:29.710843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:09:50.696827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554026,"stat_sum":{"num_bytes":21382832,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":25,"num_read":14345,"num_read_kb":34518,"num_write":383,"num_write_kb":3566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":241,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.146195+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:17:29.146195+0000","last_peered":"2024-09-21T23:17:29.146195+0000","last_clean":"2024-09-21T23:17:29.146195+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:17:29.146195+0000","last_undegraded":"2024-09-21T23:17:29.146195+0000","last_fullsized":"2024-09-21T23:17:29.146195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":326,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089202+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:17:26.089202+0000","last_peered":"2024-09-21T23:17:26.089202+0000","last_clean":"2024-09-21T23:17:26.089202+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:17:26.089202+0000","last_undegraded":"2024-09-21T23:17:26.089202+0000","last_fullsized":"2024-09-21T23:17:26.089202+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"122'806","reported_seq":11839,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120524+0000","last_change":"2024-09-21T23:16:36.047835+0000","last_active":"2024-09-21T23:17:29.120524+0000","last_peered":"2024-09-21T23:17:29.120524+0000","last_clean":"2024-09-21T23:17:29.120524+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:17:29.120524+0000","last_undegraded":"2024-09-21T23:17:29.120524+0000","last_fullsized":"2024-09-21T23:17:29.120524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:58:51.636536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041779700000000003,"stat_sum":{"num_bytes":36265277,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":12819,"num_read_kb":24856,"num_write":697,"num_write_kb":2932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"124'449","reported_seq":4074,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.107417+0000","last_change":"2024-09-21T23:16:36.050207+0000","last_active":"2024-09-21T23:17:29.107417+0000","last_peered":"2024-09-21T23:17:29.107417+0000","last_clean":"2024-09-21T23:17:29.107417+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:17:29.107417+0000","last_undegraded":"2024-09-21T23:17:29.107417+0000","last_fullsized":"2024-09-21T23:17:29.107417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:12.041299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072471600000000003,"stat_sum":{"num_bytes":16293095,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":23,"num_read":5277,"num_read_kb":11757,"num_write":363,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"125'250","reported_seq":3195,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.107160+0000","last_change":"2024-09-21T23:16:18.775878+0000","last_active":"2024-09-21T23:17:29.107160+0000","last_peered":"2024-09-21T23:17:29.107160+0000","last_clean":"2024-09-21T23:17:29.107160+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:17:29.107160+0000","last_undegraded":"2024-09-21T23:17:29.107160+0000","last_fullsized":"2024-09-21T23:17:29.107160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:00:42.115143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021264,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":2653,"num_read_kb":14551,"num_write":228,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"124'645","reported_seq":7385,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.120710+0000","last_change":"2024-09-21T23:16:18.770253+0000","last_active":"2024-09-21T23:17:29.120710+0000","last_peered":"2024-09-21T23:17:29.120710+0000","last_clean":"2024-09-21T23:17:29.120710+0000","last_became_active":"2024-09-21T23:16:18.768151+0000","last_became_peered":"2024-09-21T23:16:18.768151+0000","last_unstale":"2024-09-21T23:17:29.120710+0000","last_undegraded":"2024-09-21T23:17:29.120710+0000","last_fullsized":"2024-09-21T23:17:29.120710+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:28:12.854806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605686,"stat_sum":{"num_bytes":16748605,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":9758,"num_read_kb":19499,"num_write":599,"num_write_kb":1873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":541758,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"122'416","reported_seq":3544,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.845656+0000","last_change":"2024-09-21T23:16:06.210449+0000","last_active":"2024-09-21T23:17:29.845656+0000","last_peered":"2024-09-21T23:17:29.845656+0000","last_clean":"2024-09-21T23:17:29.845656+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:17:29.845656+0000","last_undegraded":"2024-09-21T23:17:29.845656+0000","last_fullsized":"2024-09-21T23:17:29.845656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:11:00.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014386119999999999,"stat_sum":{"num_bytes":22625325,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":4540,"num_read_kb":8546,"num_write":343,"num_write_kb":2118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"125'378","reported_seq":4474,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.641713+0000","last_change":"2024-09-21T23:16:36.052262+0000","last_active":"2024-09-21T23:17:29.641713+0000","last_peered":"2024-09-21T23:17:29.641713+0000","last_clean":"2024-09-21T23:17:29.641713+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:17:29.641713+0000","last_undegraded":"2024-09-21T23:17:29.641713+0000","last_fullsized":"2024-09-21T23:17:29.641713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:08.837464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7152531,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":5170,"num_read_kb":26289,"num_write":458,"num_write_kb":1491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"125'339","reported_seq":3296,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.703809+0000","last_change":"2024-09-21T23:15:03.322692+0000","last_active":"2024-09-21T23:17:29.703809+0000","last_peered":"2024-09-21T23:17:29.703809+0000","last_clean":"2024-09-21T23:17:29.703809+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:17:29.703809+0000","last_undegraded":"2024-09-21T23:17:29.703809+0000","last_fullsized":"2024-09-21T23:17:29.703809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:52.937174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022811810000000002,"stat_sum":{"num_bytes":10949214,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":20,"num_read":3971,"num_read_kb":12065,"num_write":327,"num_write_kb":1407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"123'366","reported_seq":5353,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.892040+0000","last_change":"2024-09-21T23:16:06.212694+0000","last_active":"2024-09-21T23:17:29.892040+0000","last_peered":"2024-09-21T23:17:29.892040+0000","last_clean":"2024-09-21T23:17:29.892040+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:17:29.892040+0000","last_undegraded":"2024-09-21T23:17:29.892040+0000","last_fullsized":"2024-09-21T23:17:29.892040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:35:32.783431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020967389999999998,"stat_sum":{"num_bytes":38361721,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":6806,"num_read_kb":27872,"num_write":339,"num_write_kb":3792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"124'597","reported_seq":3878,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.431655+0000","last_change":"2024-09-21T23:16:06.210101+0000","last_active":"2024-09-21T23:17:29.431655+0000","last_peered":"2024-09-21T23:17:29.431655+0000","last_clean":"2024-09-21T23:17:29.431655+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:17:29.431655+0000","last_undegraded":"2024-09-21T23:17:29.431655+0000","last_fullsized":"2024-09-21T23:17:29.431655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:27.363387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017007789999999999,"stat_sum":{"num_bytes":18428009,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":4521,"num_read_kb":17346,"num_write":803,"num_write_kb":3914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"124'448","reported_seq":4669,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.691427+0000","last_change":"2024-09-21T23:16:06.214833+0000","last_active":"2024-09-21T23:17:29.691427+0000","last_peered":"2024-09-21T23:17:29.691427+0000","last_clean":"2024-09-21T23:17:29.691427+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:17:29.691427+0000","last_undegraded":"2024-09-21T23:17:29.691427+0000","last_fullsized":"2024-09-21T23:17:29.691427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:50:08.266006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20196939,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":25,"num_read":5137,"num_read_kb":24636,"num_write":599,"num_write_kb":3268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"122'633","reported_seq":21156,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-21T23:17:26.089336+0000","last_change":"2024-09-21T23:16:18.769949+0000","last_active":"2024-09-21T23:17:26.089336+0000","last_peered":"2024-09-21T23:17:26.089336+0000","last_clean":"2024-09-21T23:17:26.089336+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:17:26.089336+0000","last_undegraded":"2024-09-21T23:17:26.089336+0000","last_fullsized":"2024-09-21T23:17:26.089336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:22.727972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062084,"stat_sum":{"num_bytes":37089895,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":23764,"num_read_kb":44695,"num_write":604,"num_write_kb":3041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"123'405","reported_seq":9316,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T23:17:29.146491+0000","last_change":"2024-09-21T23:16:36.050076+0000","last_active":"2024-09-21T23:17:29.146491+0000","last_peered":"2024-09-21T23:17:29.146491+0000","last_clean":"2024-09-21T23:17:29.146491+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:17:29.146491+0000","last_undegraded":"2024-09-21T23:17:29.146491+0000","last_fullsized":"2024-09-21T23:17:29.146491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:13:14.008626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12269201,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":9261,"num_read_kb":21753,"num_write":475,"num_write_kb":2020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":880956984,"num_objects":1503,"num_object_clones":405,"num_object_copies":3006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1503,"num_whiteouts":658,"num_read":277419,"num_read_kb":898915,"num_write":17352,"num_write_kb":165966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2034,"num_bytes_recovered":22712186,"num_keys_recovered":282,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":209825792,"data_stored":305401592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":15666,"ondisk_log_size":15666,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112560,"kb_used_data":54980,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94259280,"statfs":{"total":96636764160,"available":96521502720,"internally_reserved":0,"allocated":56299520,"data_stored":87495795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607589,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135672,"kb_used_data":78112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94236168,"statfs":{"total":96636764160,"available":96497836032,"internally_reserved":0,"allocated":79986688,"data_stored":121042189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607589,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607589,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607590,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121160,"kb_used_data":53876,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94250680,"statfs":{"total":96636764160,"available":96512696320,"internally_reserved":0,"allocated":55169024,"data_stored":70989180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52988,"kb_used_data":24800,"kb_used_omap":5,"kb_used_meta":28154,"kb_avail":94318852,"statfs":{"total":96636764160,"available":96582504448,"internally_reserved":0,"allocated":25395200,"data_stored":29442227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":28829989},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22962176,"data_stored":27889589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53088256,"data_stored":69788136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":78503936,"data_stored":120431532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":54812672,"data_stored":86885119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:17:31.855 INFO:tasks.thrashosds.thrasher:Setting 3.b to [0, 4] 2024-09-21T23:17:31.855 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.b', '0', '4'] 2024-09-21T23:17:31.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.b 0 4 2024-09-21T23:17:31.867 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:31.867 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:31.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:31.939 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:31.936+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:32.039 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:32.039+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.311040946999995, 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:32.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310995689999999, 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:32.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310946068000007, 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310916372999998, 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:32.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310883445000002, 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310849149000006, 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:32.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:32.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310824643000004, 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.100 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:32.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:32.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:32.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.106 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310796652999997, 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310773624999996, 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310752115, 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:32.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:32.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:32.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310716573999997, 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.120 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:32.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310688936999995, 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:32.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:32.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310653772999999, 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310618742000003, 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:32.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:32.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310583356999999, 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:32.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:32.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:32.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:32.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310543175000006, 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.143 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310506047000004, 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:32.147 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:32.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:32.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310465972000003, 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:32.156 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310441428000004, 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:32.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:32.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.310403643000001, 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:32.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.166 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:32.166 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:17:32.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:32.168 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:32.164+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:32.268 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:32.267+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:32.341 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:32.341 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:32.341 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:32.342 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:32.357 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:32.369 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:32.368+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:32.470 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:32.468+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:32.520 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:32.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:32.521 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:32.521 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:32.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:32.571 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:32.568+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:32.666 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.740278356, 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:32.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.739257509, 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:32.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:32.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.739197836, 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:32.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:32.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73910786, 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:32.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73905795, 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:32.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:32.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73900181, 2024-09-21T23:17:32.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:32.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:32.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73894802700001, 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:32.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:32.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.703 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73891575099999, 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:32.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:32.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738885471, 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:32.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:32.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73885581499999, 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:32.714 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:32.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738825848, 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:32.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.724 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738748019, 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.729 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73870031200001, 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:32.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:32.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.73867011599999, 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.735 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:32.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738640199, 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:32.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:32.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.744 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738610399, 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:32.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:32.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738580466, 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:32.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738515503, 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:32.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:32.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:32.764 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738485027, 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:32.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:32.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.738455863, 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:32.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:32.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:32.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:32.776 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:32.777 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:32.778 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:32.779 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:32.776+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:32.878 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:32.879+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:32.910 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:32.910 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:32.910 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:32.910 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:32.924 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:32.979 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:32.976+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:33.002 INFO:teuthology.orchestra.run.smithi170.stdout:5454 read 0x957a400 thru 0x957e5cf (0x41d0 bytes) 2024-09-21T23:17:33.003 INFO:teuthology.orchestra.run.smithi170.stdout:5456 read 0x7a38800 thru 0x7a48229 (0xfa2a bytes) 2024-09-21T23:17:33.004 INFO:teuthology.orchestra.run.smithi170.stdout:5457 write 0x211d200 thru 0x21278c1 (0xa6c2 bytes) 2024-09-21T23:17:33.006 INFO:teuthology.orchestra.run.smithi170.stdout:5458 clone 37 order 20 su 16384 sc 6 2024-09-21T23:17:33.045 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:33.045 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:33.046 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:33.046 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:33.046 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:33.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:33.080 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:33.079+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:33.127 INFO:teuthology.orchestra.run.smithi053.stderr:set 3.b pg_upmap mapping to [0,4] 2024-09-21T23:17:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:17:33.207 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:33.207+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 167.009994439, 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.231 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:33.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:33.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.497589156, 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.238 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:33.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:33.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.49752425299999, 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.497377814, 2024-09-21T23:17:33.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:33.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.497308699, 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.254 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:33.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:33.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:33.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.49669637400001, 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.260 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:33.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.496491969, 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.266 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:33.271 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.49644918300001, 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:33.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:33.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:33.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:33.276 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.105490895, 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:33.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:33.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:33.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.282 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.10540438500001, 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:33.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:33.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:33.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.105327496, 2024-09-21T23:17:33.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:33.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.10512265599999, 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.295 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:33.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:33.300 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.10503437400001, 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.306 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.104903546, 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:33.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:33.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.104857893, 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:33.312 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:33.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.10481946600001, 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:33.318 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:33.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.10469000399999, 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.324 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:33.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:33.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:33.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:33.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.10466836200001, 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.331 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:33.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:33.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.10464831799999, 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.337 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:33.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:33.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 166.104564326, 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:33.344 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:33.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:33.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:33.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:33.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:33.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:33.351 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:33.352 INFO:teuthology.orchestra.run.smithi170.stdout:truncating image image_client.0-clone36 from 0xd1b0800 (overlap 0xd1b0800) to 0x92c8400 2024-09-21T23:17:33.353 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:33.353 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:33.348+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:33.453 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:33.452+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:33.523 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:33.524 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:33.524 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:33.524 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:33.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:33.538 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:33.582 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:33.580+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:33.655 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:33.655+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:33.686 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:33.687 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:33.703 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:33.703 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:33.703 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:33.703 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:33.703 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:33.704 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:33.704 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:33.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:33.720 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:33.720 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:33.740 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:33.740 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:33.757 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:33.757 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:33.761 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:33.760+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:33.775 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:33.776 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:33.839 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:33.839 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:33.853 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:33.856 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:33.855+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:33.957 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:33.955+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:34.015 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:34.016 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:34.032 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:34.078 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:34.076+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:34.156 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:34.157 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:34.158 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:34.155+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:34.171 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:34.262 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:34.260+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:34.342 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:34.342 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:34.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:34.398 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:34.396+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:34.460 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:34.456+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:34.492 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:34.492 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:34.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:34.561 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:34.559+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:34.632 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:34.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:34.663 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:34.660+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:34.763 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:34.763+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:34.820 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:34.820 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:34.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:34.879 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:34.879+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:34.965 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:34.963+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:35.011 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:35.011 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:35.026 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:35.067 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:35.067+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:35.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:17:35.167 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:35.167+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:35.196 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:35.197 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:35.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:35.303 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:35.303+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:35.368 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:35.367+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:35.386 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:35.387 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:35.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:35.470 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:35.468+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:35.569 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:35.569 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:35.571 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:35.568+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:35.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:35.672 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:35.668+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:35.710 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:35.710 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:35.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:35.773 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:35.771+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:35.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:35.855 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:35.855 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:35.867 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-21T23:17:35.872 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:35.873 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:35.868+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:35.934 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:35.934 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:35.950 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:35.951 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:35.968 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:35.969 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:35.974 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:35.971+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:35.988 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:35.988 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:36.004 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:36.005 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:36.008 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:36.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:36.029 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:36.029 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:36.091 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:36.091+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:36.143 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:17:36.146 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:36.146 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:36.146 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:36.146 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:36.146 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:36.163 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:36.219 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:36.219+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:36.277 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:36.275+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:36.285 INFO:teuthology.orchestra.run.smithi170.stdout:5459 write 0xb31b600 thru 0xb321fe5 (0x69e6 bytes) 2024-09-21T23:17:36.286 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 170.066018149, 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.287 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:36.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:36.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.55361286600001, 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.294 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:36.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:36.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:36.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.553547963, 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.300 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:36.305 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.55340152400001, 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.306 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:36.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:36.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.312 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.55333240900001, 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:36.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:36.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:36.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.55272008399999, 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:36.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:36.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.324 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.55251567900001, 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:36.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:36.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.330 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.55247289299999, 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:36.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:36.336 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16151460500001, 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.337 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:36.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:36.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:36.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16142809499999, 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.343 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:36.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:36.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:36.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16135120600001, 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.349 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:36.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:36.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.161146366, 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.355 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16105808399999, 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.361 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:36.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:36.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:36.367 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16092725600001, 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:36.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:36.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.373 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16088160300001, 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:36.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.379 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16084317599999, 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:36.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:36.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.160713714, 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.386 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:36.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:36.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:36.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:36.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.393 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16069207199999, 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.394 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:36.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:36.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16067202799999, 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:36.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:36.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 169.16058803600001, 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:36.406 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:36.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.412 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:36.414 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:36.415 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:36.412+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:36.415 INFO:teuthology.orchestra.run.smithi170.stdout:5462 punch from 0xb8b4400 to 0xb8bfe00, (0xba00 bytes) 2024-09-21T23:17:36.415 INFO:teuthology.orchestra.run.smithi170.stdout:5465 write 0x5965c00 thru 0x596ada2 (0x51a3 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5467 read 0x5ff3200 thru 0x5ffb697 (0x8498 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5468 read 0x8589600 thru 0x858b885 (0x2286 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5469 write 0x3c9ca00 thru 0x3ca737c (0xa97d bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5470 punch from 0xa803000 to 0xa80cc00, (0x9c00 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5472 read 0x8293a00 thru 0x82a21c8 (0xe7c9 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5473 read 0xbda4800 thru 0xbdaac91 (0x6492 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5476 punch from 0xdee800 to 0xdf4c00, (0x6400 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5477 punch from 0xc106600 to 0xc10d800, (0x7200 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5478 punch from 0xbffe800 to 0xc006e00, (0x8600 bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5479 read 0xbc56a00 thru 0xbc577e9 (0xdea bytes) 2024-09-21T23:17:36.416 INFO:teuthology.orchestra.run.smithi170.stdout:5481 punch from 0x8d55200 to 0x8d5da00, (0x8800 bytes) 2024-09-21T23:17:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:5482 trunc from 0xd1b0800 to 0x7bcbe00 2024-09-21T23:17:36.514 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:36.512+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:36.595 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:36.595 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:36.595 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:36.595 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:36.609 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:36.654 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:36.652+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:36.717 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:36.719+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:36.776 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:36.776 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:36.776 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:36.776 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:36.777 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:36.790 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:36.817 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:36.819+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:36.918 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.963270929, 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.919 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:36.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.925 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.963003945, 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:36.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.931 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.3506754, 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:36.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.350632700999995, 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.936 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.940 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.350594080999997, 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:36.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:36.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.350562209000003, 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:36.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:36.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.350528984999997, 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:36.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.350493909999997, 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.954 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.350448951999994, 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:36.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:36.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.349996884999996, 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.963 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:36.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.967 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.349282406, 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:36.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:36.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.349241336000006, 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.972 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:36.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:36.976 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:36.916+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.977 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.348784218000006, 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.978 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.348547538999995, 2024-09-21T23:17:36.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:36.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:36.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.986 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.348131957999996, 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.987 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:36.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:36.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.347454298000002, 2024-09-21T23:17:36.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:36.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:36.995 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.346963009000007, 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:36.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:36.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:36.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:36.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:37.000 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.346706111000003, 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.004 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.637515012000001, 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:37.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.634636311000001, 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.009 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:17:37.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:17:37.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:17:37.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:17:37.015 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.015 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.015 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.015 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.015 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.015 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:37.015 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:37.019 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:37.019+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:37.120 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:37.116+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:37.148 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:37.148 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:37.148 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:37.148 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:37.164 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:37.192 INFO:teuthology.orchestra.run.smithi170.stdout:5484 write 0x3020e00 thru 0x302d014 (0xc215 bytes) 2024-09-21T23:17:37.199 INFO:teuthology.orchestra.run.smithi170.stdout:5487 read 0x3076a00 thru 0x3078c7b (0x227c bytes) 2024-09-21T23:17:37.200 INFO:teuthology.orchestra.run.smithi170.stdout:5490 punch from 0xb15800 to 0xb1e400, (0x8c00 bytes) 2024-09-21T23:17:37.208 INFO:teuthology.orchestra.run.smithi170.stdout:5493 write 0x136b600 thru 0x136e0c7 (0x2ac8 bytes) 2024-09-21T23:17:37.217 INFO:teuthology.orchestra.run.smithi170.stdout:5496 read 0x3374400 thru 0x3380484 (0xc085 bytes) 2024-09-21T23:17:37.231 INFO:teuthology.orchestra.run.smithi170.stdout:5498 read 0x63a9400 thru 0x63b72b1 (0xdeb2 bytes) 2024-09-21T23:17:37.236 INFO:teuthology.orchestra.run.smithi170.stdout:5501 trunc from 0x7bcbe00 to 0x78c1200 2024-09-21T23:17:37.249 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:37.248+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:37.299 INFO:teuthology.orchestra.run.smithi170.stdout:5503 trunc from 0x78c1200 to 0x5d69400 2024-09-21T23:17:37.322 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:37.319+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:37.344 INFO:teuthology.orchestra.run.smithi170.stdout:5504 read 0x2b38e00 thru 0x2b40bee (0x7def bytes) 2024-09-21T23:17:37.345 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:37.345 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:37.345 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:37.345 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:37.345 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:37.347 INFO:teuthology.orchestra.run.smithi170.stdout:5505 read 0x25e7600 thru 0x25f5827 (0xe228 bytes) 2024-09-21T23:17:37.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:37.412 INFO:teuthology.orchestra.run.smithi170.stdout:5508 trunc from 0x5d69400 to 0x8d3600 2024-09-21T23:17:37.445 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:37.444+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:37.466 INFO:teuthology.orchestra.run.smithi170.stdout:5509 read 0x4d6e00 thru 0x4dec3d (0x7e3e bytes) 2024-09-21T23:17:37.504 INFO:teuthology.orchestra.run.smithi170.stdout:5511 write 0x84aea00 thru 0x84b5ab5 (0x70b6 bytes) 2024-09-21T23:17:37.505 INFO:teuthology.orchestra.run.smithi170.stdout:5512 write 0x1d23a00 thru 0x1d282ce (0x48cf bytes) 2024-09-21T23:17:37.506 INFO:teuthology.orchestra.run.smithi170.stdout:5513 read 0x5e79600 thru 0x5e86d23 (0xd724 bytes) 2024-09-21T23:17:37.506 INFO:teuthology.orchestra.run.smithi170.stdout:5515 read 0x2127200 thru 0x2129fb7 (0x2db8 bytes) 2024-09-21T23:17:37.506 INFO:teuthology.orchestra.run.smithi170.stdout:5516 read 0x78cf600 thru 0x78d19fa (0x23fb bytes) 2024-09-21T23:17:37.514 INFO:teuthology.orchestra.run.smithi170.stdout:5517 trunc from 0x84b5c00 to 0x52fa800 2024-09-21T23:17:37.523 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:37.523+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:37.539 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:17:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 89.415528749999993, 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.540 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:17:37.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.545 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.411077593000002, 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:17:37.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.411050119999999, 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.411009946999997, 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.554 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:37.558 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410959177999999, 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.562 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410928595000001, 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:17:37.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410900689000002, 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.567 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:17:37.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410873468999995, 2024-09-21T23:17:37.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:17:37.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410846550000002, 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:17:37.579 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410790786999996, 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.580 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:17:37.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410757614000005, 2024-09-21T23:17:37.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:17:37.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:17:37.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410712395000004, 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:17:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410673114999994, 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:17:37.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410642628999994, 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:37.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:17:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410613335999997, 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:37.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:17:37.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:17:37.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410570993999997, 2024-09-21T23:17:37.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:37.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:17:37.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.410187798999999, 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:17:37.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:17:37.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.285340126000001, 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:17:37.615 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:17:37.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:17:37.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:17:37.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.260291166999998, 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:37.621 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:17:37.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.226857997, 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:37.627 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:17:37.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:37.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:37.631 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:37.632 INFO:teuthology.orchestra.run.smithi170.stdout:5518 read 0x52cf200 thru 0x52dcad5 (0xd8d6 bytes) 2024-09-21T23:17:37.632 INFO:teuthology.orchestra.run.smithi170.stdout:5519 write 0xed20200 thru 0xed22002 (0x1e03 bytes) 2024-09-21T23:17:37.632 INFO:teuthology.orchestra.run.smithi170.stdout:5521 trunc from 0xed22200 to 0xd9ec200 2024-09-21T23:17:37.633 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:37.633 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:37.631+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:37.652 INFO:teuthology.orchestra.run.smithi170.stdout:5525 read 0x9aab600 thru 0x9abaf51 (0xf952 bytes) 2024-09-21T23:17:37.653 INFO:teuthology.orchestra.run.smithi170.stdout:5527 read 0x9a6800 thru 0x9ab59a (0x4d9b bytes) 2024-09-21T23:17:37.654 INFO:teuthology.orchestra.run.smithi170.stdout:5530 read 0x51dc400 thru 0x51e691e (0xa51f bytes) 2024-09-21T23:17:37.656 INFO:teuthology.orchestra.run.smithi170.stdout:5533 flatten 2024-09-21T23:17:37.720 INFO:teuthology.orchestra.run.smithi170.stdout:5534 read 0x4c3a800 thru 0x4c410c3 (0x68c4 bytes) 2024-09-21T23:17:37.722 INFO:teuthology.orchestra.run.smithi170.stdout:5535 read 0x37a8400 thru 0x37b68cd (0xe4ce bytes) 2024-09-21T23:17:37.723 INFO:teuthology.orchestra.run.smithi170.stdout:5537 punch from 0xd657600 to 0xd65f600, (0x8000 bytes) 2024-09-21T23:17:37.723 INFO:teuthology.orchestra.run.smithi170.stdout:5541 read 0x2ff4200 thru 0x2ff72f5 (0x30f6 bytes) 2024-09-21T23:17:37.724 INFO:teuthology.orchestra.run.smithi170.stdout:5542 punch from 0x23bb800 to 0x23c0c00, (0x5400 bytes) 2024-09-21T23:17:37.724 INFO:teuthology.orchestra.run.smithi170.stdout:5545 write 0xa257a00 thru 0xa25db73 (0x6174 bytes) 2024-09-21T23:17:37.724 INFO:teuthology.orchestra.run.smithi170.stdout:5547 punch from 0xd5a3400 to 0xd5b2a00, (0xf600 bytes) 2024-09-21T23:17:37.725 INFO:teuthology.orchestra.run.smithi170.stdout:5548 read 0x4eba200 thru 0x4ebc13b (0x1f3c bytes) 2024-09-21T23:17:37.725 INFO:teuthology.orchestra.run.smithi170.stdout:5550 read 0x31a4000 thru 0x31af9b1 (0xb9b2 bytes) 2024-09-21T23:17:37.725 INFO:teuthology.orchestra.run.smithi170.stdout:5553 read 0xaca0000 thru 0xaca9033 (0x9034 bytes) 2024-09-21T23:17:37.725 INFO:teuthology.orchestra.run.smithi170.stdout:5556 read 0x3233800 thru 0x3241c82 (0xe483 bytes) 2024-09-21T23:17:37.726 INFO:teuthology.orchestra.run.smithi170.stdout:5557 read 0x65a5000 thru 0x65ace46 (0x7e47 bytes) 2024-09-21T23:17:37.726 INFO:teuthology.orchestra.run.smithi170.stdout:5559 read 0x230f800 thru 0x2314e3d (0x563e bytes) 2024-09-21T23:17:37.726 INFO:teuthology.orchestra.run.smithi170.stdout:5560 punch from 0x824800 to 0x825000, (0x800 bytes) 2024-09-21T23:17:37.726 INFO:teuthology.orchestra.run.smithi170.stdout:5564 read 0x2022c00 thru 0x202cb3d (0x9f3e bytes) 2024-09-21T23:17:37.727 INFO:teuthology.orchestra.run.smithi170.stdout:5566 punch from 0xd17e00 to 0xd18400, (0x600 bytes) 2024-09-21T23:17:37.727 INFO:teuthology.orchestra.run.smithi170.stdout:5567 write 0x2667400 thru 0x26720af (0xacb0 bytes) 2024-09-21T23:17:37.733 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:37.731+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:37.738 INFO:teuthology.orchestra.run.smithi170.stdout:5568 trunc from 0xd9ec200 to 0x82a2200 2024-09-21T23:17:37.785 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:37.785 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:37.786 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:37.786 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:37.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:37.802 INFO:teuthology.orchestra.run.smithi170.stdout:5569 trunc from 0x82a2200 to 0x5192000 2024-09-21T23:17:37.825 INFO:teuthology.orchestra.run.smithi170.stdout:5572 punch from 0xab1800 to 0xac0200, (0xea00 bytes) 2024-09-21T23:17:37.827 INFO:teuthology.orchestra.run.smithi170.stdout:5573 read 0x4f33200 thru 0x4f3d8a8 (0xa6a9 bytes) 2024-09-21T23:17:37.827 INFO:teuthology.orchestra.run.smithi170.stdout:5574 punch from 0x1750600 to 0x1757600, (0x7000 bytes) 2024-09-21T23:17:37.828 INFO:teuthology.orchestra.run.smithi170.stdout:5575 read 0x38ed000 thru 0x38f8165 (0xb166 bytes) 2024-09-21T23:17:37.843 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:37.843+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:37.865 INFO:teuthology.orchestra.run.smithi170.stdout:5577 write 0xcb77600 thru 0xcb7b3b3 (0x3db4 bytes) 2024-09-21T23:17:37.868 INFO:teuthology.orchestra.run.smithi170.stdout:5580 write 0x3d2e000 thru 0x3d34979 (0x697a bytes) 2024-09-21T23:17:37.876 INFO:teuthology.orchestra.run.smithi170.stdout:5582 trunc from 0xcb7b400 to 0x7b35200 2024-09-21T23:17:37.916 INFO:teuthology.orchestra.run.smithi170.stdout:5586 write 0x297600 thru 0x29d2e1 (0x5ce2 bytes) 2024-09-21T23:17:37.921 INFO:teuthology.orchestra.run.smithi170.stdout:5587 punch from 0xc5c400 to 0xc5ca00, (0x600 bytes) 2024-09-21T23:17:37.921 INFO:teuthology.orchestra.run.smithi170.stdout:5589 write 0x66a5800 thru 0x66a61f4 (0x9f5 bytes) 2024-09-21T23:17:37.921 INFO:teuthology.orchestra.run.smithi170.stdout:5590 write 0x8cba00 thru 0x8d6764 (0xad65 bytes) 2024-09-21T23:17:37.935 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:37.932+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:37.935 INFO:teuthology.orchestra.run.smithi170.stdout:5591 trunc from 0x7b35200 to 0xc29b600 2024-09-21T23:17:37.959 INFO:teuthology.orchestra.run.smithi170.stdout:5592 punch from 0x83fb600 to 0x83ffc00, (0x4600 bytes) 2024-09-21T23:17:37.961 INFO:teuthology.orchestra.run.smithi170.stdout:5593 read 0x5856600 thru 0x5862900 (0xc301 bytes) 2024-09-21T23:17:37.961 INFO:teuthology.orchestra.run.smithi170.stdout:5594 punch from 0x36cde00 to 0x36d9200, (0xb400 bytes) 2024-09-21T23:17:37.961 INFO:teuthology.orchestra.run.smithi170.stdout:5595 read 0x54d5e00 thru 0x54e182c (0xba2d bytes) 2024-09-21T23:17:37.961 INFO:teuthology.orchestra.run.smithi170.stdout:5596 write 0xa04a200 thru 0xa04b0ac (0xead bytes) 2024-09-21T23:17:37.962 INFO:teuthology.orchestra.run.smithi170.stdout:5597 read 0xb4ab200 thru 0xb4adb62 (0x2963 bytes) 2024-09-21T23:17:37.969 INFO:teuthology.orchestra.run.smithi170.stdout:5601 trunc from 0xc29b600 to 0x8407400 2024-09-21T23:17:37.972 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:37.972 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:37.972 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:37.972 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:37.972 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:37.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:38.007 INFO:teuthology.orchestra.run.smithi170.stdout:5602 trunc from 0x8407400 to 0x6f97800 2024-09-21T23:17:38.035 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:38.035+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:38.039 INFO:teuthology.orchestra.run.smithi170.stdout:5603 punch from 0x4108600 to 0x4109600, (0x1000 bytes) 2024-09-21T23:17:38.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:38.061 INFO:teuthology.orchestra.run.smithi170.stdout:5605 write 0x72d0800 thru 0x72dcc3f (0xc440 bytes) 2024-09-21T23:17:38.063 INFO:teuthology.orchestra.run.smithi170.stdout:5606 write 0x165f200 thru 0x16658ec (0x66ed bytes) 2024-09-21T23:17:38.114 INFO:teuthology.orchestra.run.smithi170.stdout:5607 write 0xb6ace00 thru 0xb6b82df (0xb4e0 bytes) 2024-09-21T23:17:38.116 INFO:teuthology.orchestra.run.smithi170.stdout:5608 write 0x1d7ca00 thru 0x1d8aca9 (0xe2aa bytes) 2024-09-21T23:17:38.121 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:38.121 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.961878222999999, 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:38.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:38.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:38.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 91.896623465000005, 2024-09-21T23:17:38.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:38.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:38.132 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 86.499281332999999, 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:38.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 86.495913643999998, 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:38.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:38.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 86.491804063999993, 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:38.141 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:38.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 86.488011142999994, 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:38.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:38.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.424095265000005, 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.150 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.424068464000001, 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.424042299000007, 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.424013849000005, 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.423923141000003, 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.423891561999994, 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.423853871999995, 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.423823482000003, 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.423780065000003, 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.423745440000005, 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.420491940000005, 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.420457357999993, 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 80.204712959000005, 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.595737763, 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:17:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:17:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:38.216 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:38.217 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:38.217 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:38.217 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-21T23:17:38.217 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:17:38.217 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:17:38.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-21T23:17:38.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:17:38.218 INFO:teuthology.orchestra.run.smithi170.stdout:5611 trunc from 0xb6b8400 to 0x17fda00 2024-09-21T23:17:38.219 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:38.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:38.220 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:38.220 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:38.221 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:38.215+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:38.231 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:38.231 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:38.247 INFO:teuthology.orchestra.run.smithi170.stdout:5613 write 0x679d600 thru 0x67a30af (0x5ab0 bytes) 2024-09-21T23:17:38.248 INFO:teuthology.orchestra.run.smithi170.stdout:5614 write 0x538be00 thru 0x539a4d9 (0xe6da bytes) 2024-09-21T23:17:38.251 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:38.252 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:38.253 INFO:teuthology.orchestra.run.smithi170.stdout:5615 trunc from 0x67a3200 to 0x4993a00 2024-09-21T23:17:38.288 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:38.288 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:38.295 INFO:teuthology.orchestra.run.smithi170.stdout:5617 read 0x878400 thru 0x879b48 (0x1749 bytes) 2024-09-21T23:17:38.298 INFO:teuthology.orchestra.run.smithi170.stdout:5618 punch from 0x18f2c00 to 0x18f8e00, (0x6200 bytes) 2024-09-21T23:17:38.307 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:38.307 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:38.318 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:38.315+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:38.325 INFO:teuthology.orchestra.run.smithi170.stdout:5619 write 0x635d800 thru 0x635ec5f (0x1460 bytes) 2024-09-21T23:17:38.354 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:38.354 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T23:17:38.354 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:38.354 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:38.354 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:38.365 INFO:teuthology.orchestra.run.smithi170.stdout:5622 write 0xcbf7e00 thru 0xcbff5a1 (0x77a2 bytes) 2024-09-21T23:17:38.367 INFO:teuthology.orchestra.run.smithi170.stdout:5623 read 0x6165a00 thru 0x616629b (0x89c bytes) 2024-09-21T23:17:38.368 INFO:teuthology.orchestra.run.smithi170.stdout:5625 write 0xa476800 thru 0xa47ac8d (0x448e bytes) 2024-09-21T23:17:38.368 INFO:teuthology.orchestra.run.smithi170.stdout:5626 punch from 0x3cfdc00 to 0x3d05c00, (0x8000 bytes) 2024-09-21T23:17:38.373 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:38.399 INFO:teuthology.orchestra.run.smithi170.stdout:5628 write 0xd99a600 thru 0xd9a0ce8 (0x66e9 bytes) 2024-09-21T23:17:38.400 INFO:teuthology.orchestra.run.smithi170.stdout:5629 punch from 0x7d1600 to 0x7d3e00, (0x2800 bytes) 2024-09-21T23:17:38.401 INFO:teuthology.orchestra.run.smithi170.stdout:5630 read 0x4464600 thru 0x44651c9 (0xbca bytes) 2024-09-21T23:17:38.401 INFO:teuthology.orchestra.run.smithi170.stdout:5631 write 0xa3e3200 thru 0xa3e5609 (0x240a bytes) 2024-09-21T23:17:38.402 INFO:teuthology.orchestra.run.smithi170.stdout:5632 write 0x75b0000 thru 0x75bd6fc (0xd6fd bytes) 2024-09-21T23:17:38.402 INFO:teuthology.orchestra.run.smithi170.stdout:5633 punch from 0x1a6e600 to 0x1a71400, (0x2e00 bytes) 2024-09-21T23:17:38.402 INFO:teuthology.orchestra.run.smithi170.stdout:5634 punch from 0xbb11200 to 0xbb1b800, (0xa600 bytes) 2024-09-21T23:17:38.403 INFO:teuthology.orchestra.run.smithi170.stdout:5640 punch from 0x5bb0000 to 0x5bb2a00, (0x2a00 bytes) 2024-09-21T23:17:38.403 INFO:teuthology.orchestra.run.smithi170.stdout:5641 read 0xc4dc000 thru 0xc4e3cf2 (0x7cf3 bytes) 2024-09-21T23:17:38.403 INFO:teuthology.orchestra.run.smithi170.stdout:5645 read 0xbbb1600 thru 0xbbb7112 (0x5b13 bytes) 2024-09-21T23:17:38.403 INFO:teuthology.orchestra.run.smithi170.stdout:5646 punch from 0x3bfb400 to 0x3c01200, (0x5e00 bytes) 2024-09-21T23:17:38.404 INFO:teuthology.orchestra.run.smithi170.stdout:5650 read 0x345f600 thru 0x346e444 (0xee45 bytes) 2024-09-21T23:17:38.404 INFO:teuthology.orchestra.run.smithi170.stdout:5652 read 0xa360800 thru 0xa360ab9 (0x2ba bytes) 2024-09-21T23:17:38.404 INFO:teuthology.orchestra.run.smithi170.stdout:5654 read 0xbde8e00 thru 0xbdf0543 (0x7744 bytes) 2024-09-21T23:17:38.405 INFO:teuthology.orchestra.run.smithi170.stdout:5655 punch from 0x366d000 to 0x3679600, (0xc600 bytes) 2024-09-21T23:17:38.405 INFO:teuthology.orchestra.run.smithi170.stdout:5656 punch from 0xb198200 to 0xb19d800, (0x5600 bytes) 2024-09-21T23:17:38.405 INFO:teuthology.orchestra.run.smithi170.stdout:5657 write 0x6058600 thru 0x605dbd1 (0x55d2 bytes) 2024-09-21T23:17:38.410 INFO:teuthology.orchestra.run.smithi170.stdout:5660 trunc from 0xd9a0e00 to 0xabdea00 2024-09-21T23:17:38.419 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:38.416+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:38.462 INFO:teuthology.orchestra.run.smithi170.stdout:5661 read 0x269cc00 thru 0x269dd57 (0x1158 bytes) 2024-09-21T23:17:38.465 INFO:teuthology.orchestra.run.smithi170.stdout:5663 read 0x7a38c00 thru 0x7a3f5fe (0x69ff bytes) 2024-09-21T23:17:38.493 INFO:teuthology.orchestra.run.smithi170.stdout:5664 write 0xe991000 thru 0xe9a0ce0 (0xfce1 bytes) 2024-09-21T23:17:38.494 INFO:teuthology.orchestra.run.smithi170.stdout:5665 clone 38 order 22 su 65536 sc 13 2024-09-21T23:17:38.520 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:38.516+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:38.542 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:38.542 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:38.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:38.621 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:38.620+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:38.675 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:38.675 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:38.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:38.735 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:38.735+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:38.810 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:38.810 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:38.823 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:38.823+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:38.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:38.924 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:38.923+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:38.997 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:38.997 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:39.012 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:39.055 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:39.055+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:39.127 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:39.124+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:17:39.188 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:39.188 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:39.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:39.247 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:39.247+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:39.329 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:39.324+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:39.339 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:39.339 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:39.353 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:39.437 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:39.436+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:39.515 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:39.515 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:39.530 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:39.528+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:39.531 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:39.631 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:39.628+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:39.698 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:39.699 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:39.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:39.754 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:39.752+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:39.833 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:39.831+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:39.883 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:39.884 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:39.897 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:39.935 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:39.931+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:39.994 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-21T23:17:40.022 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:40.022 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:40.035 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:40.032+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:40.036 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:40.136 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:40.135+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:40.205 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:40.205 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:40.220 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:40.266 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:40.264+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:40.330 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:40.337 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:40.336+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:40.349 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:40.349 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:40.364 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:40.450 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:40.448+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:40.498 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:40.516 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:40.530 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:40.530 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:40.532 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:40.533 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:40.538 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:40.536+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:40.546 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:40.552 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:40.552 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:40.571 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:40.571 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:40.591 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:40.591 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:40.639 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:40.636+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4963.0:602 3.e 3:72ce3e86:::rbd_object_map.10f1b1af2a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:40.724 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 92.598864122999998, 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038916801000000001, 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4963", 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/254600378", 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 602 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121457+0000", 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121458+0000", 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121461+0000", 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7699999999999999e-06 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121482+0000", 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0513000000000002e-05 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.121490+0000", 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1270000000000003e-06 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156340+0000", 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034850218000000002 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.156582+0000", 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024138999999999999 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.157531+0000", 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00094944500000000004 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.159622+0000", 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020911570000000002 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160219+0000", 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059658300000000003 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160303+0000", 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3792e-05 2024-09-21T23:17:40.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:08.160374+0000", 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1155999999999994e-05 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10127 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6158336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594412965999993, 2024-09-21T23:17:40.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039138752999999998, 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10127 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125908+0000", 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125909+0000", 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2399999999999998e-07 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125911+0000", 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-21T23:17:40.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125920+0000", 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125927+0000", 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9330000000000004e-06 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164790+0000", 2024-09-21T23:17:40.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038862997000000003 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.164908+0000", 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011798600000000001 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165047+0000", 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013908700000000001 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10128 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594385493000004, 2024-09-21T23:17:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039416083999999997, 2024-09-21T23:17:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.735 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10128 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125936+0000", 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1600000000000001e-07 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125939+0000", 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4399999999999999e-06 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125948+0000", 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9500000000000007e-06 2024-09-21T23:17:40.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125954+0000", 2024-09-21T23:17:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7939999999999997e-06 2024-09-21T23:17:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165094+0000", 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039139696000000002 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165210+0000", 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011568 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165352+0000", 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141908 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10129 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6174720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594345320000002, 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039706713999999997, 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.752 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10129 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125976+0000", 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125977+0000", 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0999999999999997e-07 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.125992+0000", 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5500000000000001e-05 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126009+0000", 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6549e-05 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126016+0000", 2024-09-21T23:17:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3000000000000004e-06 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165394+0000", 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039378141999999998 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165533+0000", 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013867999999999999 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165682+0000", 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014973300000000001 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10130 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6205440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594294551000004, 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040081022000000001, 2024-09-21T23:17:40.757 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10130 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126027+0000", 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1999999999999999e-07 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126030+0000", 2024-09-21T23:17:40.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126042+0000", 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1307e-05 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126048+0000", 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9870000000000002e-06 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165764+0000", 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039715417000000003 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.165923+0000", 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015936800000000001 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166108+0000", 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018427999999999999 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.761 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10131 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6191104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594263968000007, 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040381161999999998, 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10131 2024-09-21T23:17:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:40.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126057+0000", 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126058+0000", 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3699999999999999e-07 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126061+0000", 2024-09-21T23:17:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7800000000000001e-06 2024-09-21T23:17:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126070+0000", 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8259999999999999e-06 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126077+0000", 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9530000000000002e-06 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166172+0000", 2024-09-21T23:17:40.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040095895999999999 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166292+0000", 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011989600000000001 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166438+0000", 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145974 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.767 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10132 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594236061999993, 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040652191999999997, 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.768 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10132 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126085+0000", 2024-09-21T23:17:40.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126086+0000", 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9999999999999997e-07 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126088+0000", 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6929999999999999e-06 2024-09-21T23:17:40.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126097+0000", 2024-09-21T23:17:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6340000000000007e-06 2024-09-21T23:17:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126104+0000", 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8800000000000002e-06 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166484+0000", 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040380243000000003 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166600+0000", 2024-09-21T23:17:40.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115822 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166737+0000", 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013721999999999999 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10133 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:40.773 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594208842, 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041054054999999999, 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10133 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126112+0000", 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126113+0000", 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2699999999999999e-07 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126115+0000", 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-06 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126124+0000", 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5369999999999997e-06 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126131+0000", 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7399999999999998e-06 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166822+0000", 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040691873000000003 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.166982+0000", 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159145 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167166+0000", 2024-09-21T23:17:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018467700000000001 2024-09-21T23:17:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10134 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6238208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:40.779 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594181922999994, 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041413792999999997, 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10134 2024-09-21T23:17:40.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126139+0000", 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126140+0000", 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6400000000000002e-07 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126142+0000", 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4959999999999999e-06 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126151+0000", 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5169999999999999e-06 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126158+0000", 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9600000000000003e-06 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167245+0000", 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041087008000000001 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167389+0000", 2024-09-21T23:17:40.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000144554 2024-09-21T23:17:40.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167553+0000", 2024-09-21T23:17:40.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016359400000000001 2024-09-21T23:17:40.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.816 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10135 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6223872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594126160000002, 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041727302000000001, 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.817 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10135 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126195+0000", 2024-09-21T23:17:40.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126196+0000", 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1999999999999998e-07 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126199+0000", 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126210+0000", 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1113e-05 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126218+0000", 2024-09-21T23:17:40.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2189999999999998e-06 2024-09-21T23:17:40.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167604+0000", 2024-09-21T23:17:40.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041386135999999997 2024-09-21T23:17:40.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167740+0000", 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013677399999999999 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167922+0000", 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018177300000000001 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10136 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6254592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594092986999996, 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042050483999999999, 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10136 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126228+0000", 2024-09-21T23:17:40.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126229+0000", 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126243+0000", 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3807e-05 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126257+0000", 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.437e-05 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126264+0000", 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8959999999999997e-06 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.167974+0000", 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041710228000000002 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168115+0000", 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014041699999999999 2024-09-21T23:17:40.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168279+0000", 2024-09-21T23:17:40.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016403000000000001 2024-09-21T23:17:40.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.830 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10137 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6240256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594047767999996, 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042377053999999997, 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10137 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126273+0000", 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126274+0000", 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2300000000000001e-07 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126277+0000", 2024-09-21T23:17:40.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8169999999999999e-06 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126285+0000", 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4999999999999999e-06 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126292+0000", 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.99e-06 2024-09-21T23:17:40.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168329+0000", 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042037204000000002 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168480+0000", 2024-09-21T23:17:40.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015087 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168650+0000", 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017005000000000001 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10138 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6270976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.594008488, 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774036000000001, 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10138 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126313+0000", 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0299999999999998e-07 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126316+0000", 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126326+0000", 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9930000000000007e-06 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126333+0000", 2024-09-21T23:17:40.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.007e-06 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168740+0000", 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042406758000000003 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.168902+0000", 2024-09-21T23:17:40.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161328 2024-09-21T23:17:40.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169087+0000", 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000185117 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10139 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:40.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.593978002, 2024-09-21T23:17:40.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043129038000000001, 2024-09-21T23:17:40.845 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.845 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.845 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10139 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126343+0000", 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126344+0000", 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:40.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126346+0000", 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126357+0000", 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0419e-05 2024-09-21T23:17:40.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126364+0000", 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.844e-06 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169159+0000", 2024-09-21T23:17:40.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042795829000000001 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169306+0000", 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014605 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169472+0000", 2024-09-21T23:17:40.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016658600000000001 2024-09-21T23:17:40.851 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.851 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.851 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.851 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10140 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6287360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.593948709000003, 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043471133000000002, 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.852 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10140 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126372+0000", 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126373+0000", 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126384+0000", 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126394+0000", 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126401+0000", 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1359999999999999e-06 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169532+0000", 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043131458999999997 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169679+0000", 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014677099999999999 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169844+0000", 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016448400000000001 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.857 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10141 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6273024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.593906367000002, 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811707999999998, 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10141 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:40.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126415+0000", 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7400000000000003e-07 2024-09-21T23:17:40.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126418+0000", 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8959999999999999e-06 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126427+0000", 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.037e-06 2024-09-21T23:17:40.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126434+0000", 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9260000000000003e-06 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.169911+0000", 2024-09-21T23:17:40.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043476899999999999 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170060+0000", 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014856299999999999 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170226+0000", 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016671200000000001 2024-09-21T23:17:40.863 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.864 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.864 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.864 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:10143 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6289408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:40.864 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:40.864 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 87.593523172000005, 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043869290999999998, 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 10143 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126798+0000", 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126799+0000", 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0299999999999997e-07 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126803+0000", 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126817+0000", 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.383e-05 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.126825+0000", 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1960000000000008e-06 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170332+0000", 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043506482999999999 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170487+0000", 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015507800000000001 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:13.170667+0000", 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000180404 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.468675499, 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:17:40.870 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:17:40.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.44362654, 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:17:40.877 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:17:40.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:17:40.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:17:40.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.410193370000002, 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:17:40.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:17:40.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:40.889 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:40.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:40.891 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:40.890+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:40.992 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:40.988+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:41.029 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:41.030 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:41.030 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:41.030 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:41.045 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:41.093 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:41.088+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:41.175 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:41.175 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:41.175 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:41.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:41.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:17:41.194 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:41.190+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:41.195 INFO:teuthology.orchestra.run.smithi170.stdout:5666 write 0x1ca6a00 thru 0x1cb3b06 (0xd107 bytes) 2024-09-21T23:17:41.196 INFO:teuthology.orchestra.run.smithi170.stdout:5669 read 0xc866e00 thru 0xc870dad (0x9fae bytes) 2024-09-21T23:17:41.199 INFO:teuthology.orchestra.run.smithi170.stdout:5670 flatten 2024-09-21T23:17:41.294 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:41.290+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:41.333 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:41.333 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:41.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:41.333 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:41.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 99.172251559000003, 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:41.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:41.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.106996800999994, 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:41.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:41.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.709654669000003, 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:41.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:41.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.706286980000002, 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:41.348 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:41.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.702177399999997, 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:41.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:41.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:41.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.698384478999998, 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:41.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634468600999995, 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634441800000005, 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634415634999996, 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:41.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:41.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:41.374 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634387184999994, 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:41.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.378 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634296477000007, 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:41.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.383 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634264897999998, 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:41.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:41.387 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634227207999999, 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:41.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:41.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.392 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634196818000007, 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:41.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634153401000006, 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:41.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.634118775999994, 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.630865275999994, 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.630830693999997, 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 83.415086294999995, 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8061110989999998, 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:17:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:17:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:17:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:17:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.428 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:41.429 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:41.426+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:41.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:41.443 INFO:teuthology.orchestra.run.smithi170.stdout:5673 read 0x992c200 thru 0x99335c1 (0x73c2 bytes) 2024-09-21T23:17:41.452 INFO:teuthology.orchestra.run.smithi170.stdout:5674 trunc from 0xe9a0e00 to 0xdabd400 2024-09-21T23:17:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:5675 write 0xba07a00 thru 0xba13052 (0xb653 bytes) 2024-09-21T23:17:41.470 INFO:teuthology.orchestra.run.smithi170.stdout:5676 read 0xced9e00 thru 0xcee420c (0xa40d bytes) 2024-09-21T23:17:41.470 INFO:teuthology.orchestra.run.smithi170.stdout:5677 write 0x7067200 thru 0x7068d53 (0x1b54 bytes) 2024-09-21T23:17:41.470 INFO:teuthology.orchestra.run.smithi170.stdout:5679 read 0x56cc400 thru 0x56d6d4e (0xa94f bytes) 2024-09-21T23:17:41.470 INFO:teuthology.orchestra.run.smithi170.stdout:5680 write 0x5666e00 thru 0x566ce83 (0x6084 bytes) 2024-09-21T23:17:41.471 INFO:teuthology.orchestra.run.smithi170.stdout:5682 punch from 0x3777400 to 0x377ce00, (0x5a00 bytes) 2024-09-21T23:17:41.473 INFO:teuthology.orchestra.run.smithi170.stdout:5683 trunc from 0xdabd400 to 0xd9bae00 2024-09-21T23:17:41.500 INFO:teuthology.orchestra.run.smithi170.stdout:5684 read 0x9859600 thru 0x9863829 (0xa22a bytes) 2024-09-21T23:17:41.530 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:41.526+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:41.544 INFO:teuthology.orchestra.run.smithi170.stdout:5685 trunc from 0xd9bae00 to 0x6c57600 2024-09-21T23:17:41.588 INFO:teuthology.orchestra.run.smithi170.stdout:5686 trunc from 0x6c57600 to 0xc21d200 2024-09-21T23:17:41.595 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:41.595 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:41.595 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:41.595 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:41.608 INFO:teuthology.orchestra.run.smithi170.stdout:5687 read 0x5732c00 thru 0x5736da2 (0x41a3 bytes) 2024-09-21T23:17:41.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:41.611 INFO:teuthology.orchestra.run.smithi170.stdout:5689 write 0x7aa4200 thru 0x7aa9cca (0x5acb bytes) 2024-09-21T23:17:41.611 INFO:teuthology.orchestra.run.smithi170.stdout:5691 punch from 0x4e66a00 to 0x4e6a400, (0x3a00 bytes) 2024-09-21T23:17:41.611 INFO:teuthology.orchestra.run.smithi170.stdout:5692 read 0x7cf0e00 thru 0x7cf3231 (0x2432 bytes) 2024-09-21T23:17:41.611 INFO:teuthology.orchestra.run.smithi170.stdout:5693 read 0x859c200 thru 0x85aa581 (0xe382 bytes) 2024-09-21T23:17:41.611 INFO:teuthology.orchestra.run.smithi170.stdout:5694 read 0x1f3d600 thru 0x1f4abbf (0xd5c0 bytes) 2024-09-21T23:17:41.612 INFO:teuthology.orchestra.run.smithi170.stdout:5696 read 0x5ce6800 thru 0x5cea10c (0x390d bytes) 2024-09-21T23:17:41.613 INFO:teuthology.orchestra.run.smithi170.stdout:5700 punch from 0x118d600 to 0x1190a00, (0x3400 bytes) 2024-09-21T23:17:41.613 INFO:teuthology.orchestra.run.smithi170.stdout:5701 write 0x7a10800 thru 0x7a18f93 (0x8794 bytes) 2024-09-21T23:17:41.613 INFO:teuthology.orchestra.run.smithi170.stdout:5703 read 0x336f600 thru 0x3375926 (0x6327 bytes) 2024-09-21T23:17:41.614 INFO:teuthology.orchestra.run.smithi170.stdout:5705 write 0x64ed000 thru 0x64f2c67 (0x5c68 bytes) 2024-09-21T23:17:41.647 INFO:teuthology.orchestra.run.smithi170.stdout:5706 write 0xc810000 thru 0xc819ccb (0x9ccc bytes) 2024-09-21T23:17:41.651 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:41.650+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:41.657 INFO:teuthology.orchestra.run.smithi170.stdout:5707 trunc from 0xc819e00 to 0x95b7a00 2024-09-21T23:17:41.719 INFO:teuthology.orchestra.run.smithi170.stdout:5708 write 0xa78f000 thru 0xa794153 (0x5154 bytes) 2024-09-21T23:17:41.721 INFO:teuthology.orchestra.run.smithi170.stdout:5709 write 0x344f800 thru 0x345f04c (0xf84d bytes) 2024-09-21T23:17:41.721 INFO:teuthology.orchestra.run.smithi170.stdout:5710 read 0x7737400 thru 0x773e5dc (0x71dd bytes) 2024-09-21T23:17:41.721 INFO:teuthology.orchestra.run.smithi170.stdout:5712 punch from 0x2488800 to 0x2490c00, (0x8400 bytes) 2024-09-21T23:17:41.722 INFO:teuthology.orchestra.run.smithi170.stdout:5713 write 0xb50c00 thru 0xb51069 (0x46a bytes) 2024-09-21T23:17:41.732 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:41.730+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:41.733 INFO:teuthology.orchestra.run.smithi170.stdout:5714 read 0x5adb800 thru 0x5ae9a3a (0xe23b bytes) 2024-09-21T23:17:41.733 INFO:teuthology.orchestra.run.smithi170.stdout:5716 punch from 0x6a41800 to 0x6a4a800, (0x9000 bytes) 2024-09-21T23:17:41.734 INFO:teuthology.orchestra.run.smithi170.stdout:5718 read 0xa0e3200 thru 0xa0eaaa0 (0x78a1 bytes) 2024-09-21T23:17:41.734 INFO:teuthology.orchestra.run.smithi170.stdout:5720 punch from 0x35c1a00 to 0x35c2800, (0xe00 bytes) 2024-09-21T23:17:41.734 INFO:teuthology.orchestra.run.smithi170.stdout:5722 write 0x1cf3e00 thru 0x1cfb9f6 (0x7bf7 bytes) 2024-09-21T23:17:41.735 INFO:teuthology.orchestra.run.smithi170.stdout:5723 read 0x21a1e00 thru 0x21ae2ac (0xc4ad bytes) 2024-09-21T23:17:41.735 INFO:teuthology.orchestra.run.smithi170.stdout:5724 write 0x84e7200 thru 0x84f3086 (0xbe87 bytes) 2024-09-21T23:17:41.768 INFO:teuthology.orchestra.run.smithi170.stdout:5725 trunc from 0xa794200 to 0x2d34200 2024-09-21T23:17:41.788 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:41.789 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:41.789 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:41.789 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:41.789 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:41.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:41.848 INFO:teuthology.orchestra.run.smithi170.stdout:5728 write 0x5b14400 thru 0x5b1c481 (0x8082 bytes) 2024-09-21T23:17:41.849 INFO:teuthology.orchestra.run.smithi170.stdout:5729 read 0x5ab2600 thru 0x5abff83 (0xd984 bytes) 2024-09-21T23:17:41.851 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:41.850+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:41.857 INFO:teuthology.orchestra.run.smithi170.stdout:5730 trunc from 0x5b1c600 to 0x8d59a00 2024-09-21T23:17:41.926 INFO:teuthology.orchestra.run.smithi170.stdout:5743 write 0xc1b2000 thru 0xc1bb9f0 (0x99f1 bytes) 2024-09-21T23:17:41.927 INFO:teuthology.orchestra.run.smithi170.stdout:5745 read 0x7939a00 thru 0x793b15b (0x175c bytes) 2024-09-21T23:17:41.927 INFO:teuthology.orchestra.run.smithi170.stdout:5749 punch from 0x523f200 to 0x524d000, (0xde00 bytes) 2024-09-21T23:17:41.928 INFO:teuthology.orchestra.run.smithi170.stdout:5751 read 0x6ed4600 thru 0x6ed4e86 (0x887 bytes) 2024-09-21T23:17:41.928 INFO:teuthology.orchestra.run.smithi170.stdout:5752 read 0x4191c00 thru 0x419b327 (0x9728 bytes) 2024-09-21T23:17:41.928 INFO:teuthology.orchestra.run.smithi170.stdout:5753 read 0x9134200 thru 0x91367b0 (0x25b1 bytes) 2024-09-21T23:17:41.929 INFO:teuthology.orchestra.run.smithi170.stdout:5755 read 0x2366c00 thru 0x23710df (0xa4e0 bytes) 2024-09-21T23:17:41.933 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:41.930+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:41.939 INFO:teuthology.orchestra.run.smithi170.stdout:5756 trunc from 0xc1bba00 to 0xedce600 2024-09-21T23:17:41.959 INFO:teuthology.orchestra.run.smithi170.stdout:5757 punch from 0x8105e00 to 0x810f200, (0x9400 bytes) 2024-09-21T23:17:41.960 INFO:teuthology.orchestra.run.smithi170.stdout:5758 write 0x629c000 thru 0x62a1a01 (0x5a02 bytes) 2024-09-21T23:17:41.960 INFO:teuthology.orchestra.run.smithi170.stdout:5759 write 0xb7fd200 thru 0xb8033c2 (0x61c3 bytes) 2024-09-21T23:17:41.960 INFO:teuthology.orchestra.run.smithi170.stdout:5760 punch from 0xa925400 to 0xa930200, (0xae00 bytes) 2024-09-21T23:17:41.968 INFO:teuthology.orchestra.run.smithi170.stdout:5761 trunc from 0xedce600 to 0xb1c5e00 2024-09-21T23:17:41.983 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:41.983 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:41.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:41.983 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:41.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:41.983 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:41.984 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.221156733000001, 2024-09-21T23:17:41.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:41.984 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.984 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.984 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:41.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:41.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.985 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:41.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:41.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:41.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:41.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:41.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:41.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:41.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:41.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:41.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.992 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:41.993 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:41.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.993 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:41.993 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:41.993 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.221111476000004, 2024-09-21T23:17:41.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:41.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.995 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:41.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:41.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:41.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:41.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:41.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:41.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:41.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:41.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:41.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:41.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:41.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:41.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:41.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:41.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:42.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:42.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.002 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.002 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.002 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:42.003 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.221061853999998, 2024-09-21T23:17:42.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:42.003 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.004 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.004 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:42.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.004 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:42.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:42.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:42.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:42.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:42.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:42.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:42.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:42.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:42.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:42.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:42.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:42.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:42.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:42.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:42.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:42.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.012 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:42.013 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.221032159000004, 2024-09-21T23:17:42.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:42.013 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:42.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:42.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:42.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:42.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:42.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:42.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:42.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:42.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:42.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:42.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:42.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:42.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:42.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:42.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:42.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:42.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:42.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:42.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.022 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.022 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:42.023 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220999230999993, 2024-09-21T23:17:42.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:42.023 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.024 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:42.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.024 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:42.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:42.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:42.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:42.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:42.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:42.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:42.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:42.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:42.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:42.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:42.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:42.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:42.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:42.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220964934999998, 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:42.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220940428999995, 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:42.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:42.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220912439000003, 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220889411000002, 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220867901000005, 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220832360000003, 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220804723000001, 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220769559000004, 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220734527999994, 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220699143000004, 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220658960999998, 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220621832999996, 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220581757999994, 2024-09-21T23:17:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220557213999996, 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.220519429000007, 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.104 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:42.106 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:42.106 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:42.102+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5762 punch from 0x5374e00 to 0x5377400, (0x2600 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5764 read 0x485b400 thru 0x485c419 (0x101a bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5768 write 0x84e4c00 thru 0x84eca09 (0x7e0a bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5771 trunc from 0xb1c5e00 to 0x518f600 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5772 read 0x210ce00 thru 0x2119327 (0xc528 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5773 punch from 0x3a4f000 to 0x3a52400, (0x3400 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5774 punch from 0x28d5200 to 0x28e4800, (0xf600 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5775 read 0x3430200 thru 0x3439d27 (0x9b28 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5776 write 0x41b7c00 thru 0x41c7255 (0xf656 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5778 punch from 0x1f25600 to 0x1f33a00, (0xe400 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5779 read 0x3479c00 thru 0x34832d8 (0x96d9 bytes) 2024-09-21T23:17:42.107 INFO:teuthology.orchestra.run.smithi170.stdout:5781 trunc from 0x518f600 to 0x4440a00 2024-09-21T23:17:42.138 INFO:teuthology.orchestra.run.smithi170.stdout:5782 write 0xa53c600 thru 0xa546b0b (0xa50c bytes) 2024-09-21T23:17:42.138 INFO:teuthology.orchestra.run.smithi170.stdout:5783 read 0x969c600 thru 0x969fa12 (0x3413 bytes) 2024-09-21T23:17:42.138 INFO:teuthology.orchestra.run.smithi170.stdout:5785 write 0x2db2e00 thru 0x2db6f63 (0x4164 bytes) 2024-09-21T23:17:42.139 INFO:teuthology.orchestra.run.smithi170.stdout:5786 read 0xa07a800 thru 0xa0801e2 (0x59e3 bytes) 2024-09-21T23:17:42.139 INFO:teuthology.orchestra.run.smithi170.stdout:5787 write 0x34f4000 thru 0x34ff336 (0xb337 bytes) 2024-09-21T23:17:42.140 INFO:teuthology.orchestra.run.smithi170.stdout:5788 write 0x3bd3400 thru 0x3bda233 (0x6e34 bytes) 2024-09-21T23:17:42.140 INFO:teuthology.orchestra.run.smithi170.stdout:5790 read 0x97b3e00 thru 0x97b6bdc (0x2ddd bytes) 2024-09-21T23:17:42.170 INFO:teuthology.orchestra.run.smithi170.stdout:5791 trunc from 0xa546c00 to 0xc8f200 2024-09-21T23:17:42.189 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:17:42.205 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:42.202+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:42.263 INFO:teuthology.orchestra.run.smithi170.stdout:5792 write 0x4a58000 thru 0x4a634a8 (0xb4a9 bytes) 2024-09-21T23:17:42.288 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:42.288 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:42.288 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:42.288 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:42.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:42.306 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:42.302+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:42.310 INFO:teuthology.orchestra.run.smithi170.stdout:5794 write 0xbf55a00 thru 0xbf5b92e (0x5f2f bytes) 2024-09-21T23:17:42.338 INFO:teuthology.orchestra.run.smithi170.stdout:5796 write 0xc100a00 thru 0xc10374f (0x2d50 bytes) 2024-09-21T23:17:42.340 INFO:teuthology.orchestra.run.smithi170.stdout:5797 write 0x3d36e00 thru 0x3d3731d (0x51e bytes) 2024-09-21T23:17:42.340 INFO:teuthology.orchestra.run.smithi170.stdout:5798 write 0x584a600 thru 0x585658d (0xbf8e bytes) 2024-09-21T23:17:42.341 INFO:teuthology.orchestra.run.smithi170.stdout:5801 punch from 0x3473800 to 0x3477e00, (0x4600 bytes) 2024-09-21T23:17:42.341 INFO:teuthology.orchestra.run.smithi170.stdout:5802 read 0x8d00000 thru 0x8d0992d (0x992e bytes) 2024-09-21T23:17:42.353 INFO:teuthology.orchestra.run.smithi170.stdout:5803 trunc from 0xc103800 to 0x6d55000 2024-09-21T23:17:42.406 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:42.402+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:42.412 INFO:teuthology.orchestra.run.smithi170.stdout:5805 write 0x927b600 thru 0x9288259 (0xcc5a bytes) 2024-09-21T23:17:42.413 INFO:teuthology.orchestra.run.smithi170.stdout:5806 punch from 0x2ee800 to 0x2fae00, (0xc600 bytes) 2024-09-21T23:17:42.414 INFO:teuthology.orchestra.run.smithi170.stdout:5808 read 0x89dc800 thru 0x89e6000 (0x9801 bytes) 2024-09-21T23:17:42.414 INFO:teuthology.orchestra.run.smithi170.stdout:5813 write 0x85cb200 thru 0x85d3c34 (0x8a35 bytes) 2024-09-21T23:17:42.415 INFO:teuthology.orchestra.run.smithi170.stdout:5814 read 0x8adaa00 thru 0x8ae8729 (0xdd2a bytes) 2024-09-21T23:17:42.441 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:42.441 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:42.441 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:42.441 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:42.441 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:42.446 INFO:teuthology.orchestra.run.smithi170.stdout:5817 write 0x9d6a600 thru 0x9d72f9d (0x899e bytes) 2024-09-21T23:17:42.449 INFO:teuthology.orchestra.run.smithi170.stdout:5818 punch from 0x71d5e00 to 0x71d6600, (0x800 bytes) 2024-09-21T23:17:42.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:42.457 INFO:teuthology.orchestra.run.smithi170.stdout:5819 trunc from 0x9d73000 to 0x6676e00 2024-09-21T23:17:42.504 INFO:teuthology.orchestra.run.smithi170.stdout:5820 trunc from 0x6676e00 to 0x29ae800 2024-09-21T23:17:42.507 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:42.504+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:42.543 INFO:teuthology.orchestra.run.smithi170.stdout:5821 punch from 0xf51800 to 0xf58e00, (0x7600 bytes) 2024-09-21T23:17:42.547 INFO:teuthology.orchestra.run.smithi170.stdout:5823 read 0x17ac400 thru 0x17b1984 (0x5585 bytes) 2024-09-21T23:17:42.548 INFO:teuthology.orchestra.run.smithi170.stdout:5825 read 0x88e200 thru 0x88e395 (0x196 bytes) 2024-09-21T23:17:42.602 INFO:teuthology.orchestra.run.smithi170.stdout:5827 write 0xe180e00 thru 0xe184dcd (0x3fce bytes) 2024-09-21T23:17:42.603 INFO:teuthology.orchestra.run.smithi170.stdout:5828 read 0x323800 thru 0x33190a (0xe10b bytes) 2024-09-21T23:17:42.605 INFO:teuthology.orchestra.run.smithi170.stdout:5832 write 0xdf0e00 thru 0xdfff64 (0xf165 bytes) 2024-09-21T23:17:42.607 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:42.604+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:42.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:42.619 INFO:teuthology.orchestra.run.smithi170.stdout:5833 trunc from 0xe184e00 to 0x9215000 2024-09-21T23:17:42.653 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:42.653 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.726596376, 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.725575529, 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:42.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:42.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.725515856, 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72542588, 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:42.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72537597, 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:42.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72531983, 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:42.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:42.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72526604700001, 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:42.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:42.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:42.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72523377100001, 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.694 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.725203491, 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.699 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72517383500001, 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:42.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:42.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.725143868, 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:42.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:42.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.725066039, 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.725018332, 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:42.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:42.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:42.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:42.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:42.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72498813599999, 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.724958219, 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:42.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:42.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:42.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.729 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72492841899999, 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:42.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:42.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:42.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.724898486, 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:42.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:42.738 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.724833523, 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.739 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.743 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.72480304699999, 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:42.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:42.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.724773883, 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:42.748 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:42.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:42.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:42.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:42.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.753 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:42.753 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:42.753 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:42.753 INFO:teuthology.orchestra.run.smithi170.stdout:5834 clone 39 order 22 su 1048576 sc 12 2024-09-21T23:17:42.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:42.755 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:42.752+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:42.762 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:42.762 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:42.779 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:42.779 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:42.813 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:42.834 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:42.834 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:42.854 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:42.854 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:42.854 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:42.850+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:42.888 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:42.888 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:42.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:42.955 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:42.952+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:43.026 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:43.026 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:43.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:43.056 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:43.054+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:43.157 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:43.154+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:43.211 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:43.211 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:43.226 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:43.273 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:43.272+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:43.350 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:43.350 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:43.358 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:43.356+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:43.364 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:43.375 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:43.375 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:43.375 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T23:17:43.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-21T23:17:43.458 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:43.456+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:43.533 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:43.533 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:43.547 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:17:43.547 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:17:43.547 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:17:43.547 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:17:43.547 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:17:43.547 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:17:43.547 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:17:43.548 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:17:43.548 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:17:43.548 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:17:43.548 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:17:43.548 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:17:43.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:43.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:17:43.551 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:17:43.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-21T23:17:43.593 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:43.592+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:43.661 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:43.656+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:17:43.704 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:17:43.705 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:17:43.705 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:17:43.705 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:17:43.705 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:17:43.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:17:43.707 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:17:43.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-21T23:17:43.724 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:43.724 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:43.738 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:43.782 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:43.780+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:43.862 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:43.858+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:17:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:17:43.867 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:17:43.868 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-21T23:17:43.869 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:17:43.869 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:17:43.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-21T23:17:43.912 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:43.912 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:43.926 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:43.962 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:43.958+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:44.063 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:44.063 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:44.063 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:44.060+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:44.081 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:44.088 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-21T23:17:44.088 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:bench [] [] [] [] 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore allocator dump block 2024-09-21T23:17:44.089 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: invalid command 2024-09-21T23:17:44.092 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T23:17:44.092 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T23:17:44.164 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:44.162+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:44.215 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:17:44.258 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:44.258 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:44.265 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:44.262+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:44.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:44.365 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:44.362+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:44.398 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:44.398 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:44.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:44.499 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:44.498+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:44.567 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:44.566+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:44.596 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:44.596 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:44.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:44.668 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:44.664+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:44.768 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:44.766+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:44.783 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:44.783 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:44.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:44.869 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:44.868+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:44.872 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:44.971 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:44.968+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:44.991 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:44.992 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:45.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:45.051 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:45.051 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:45.069 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:45.070 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:45.071 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:45.070+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:45.089 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:45.114 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:45.115 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:45.140 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:45.149 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:45.149 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:45.149 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:45.166 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:45.167 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:45.167 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:45.187 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:45.182+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:45.201 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:17:45.273 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:45.268+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:45.318 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:45.318 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558768817000001, 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:45.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558723560000004, 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:45.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558673937999998, 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558644243000003, 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558611314999993, 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558577018999998, 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558552512999995, 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558524523000003, 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558501495000002, 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558479985000005, 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558444444000003, 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558416807, 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558381643000004, 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558346611999994, 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558311227000004, 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558271044999998, 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558233916999995, 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558193841999994, 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558169297999996, 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.558131513000006, 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.423 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:45.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:45.425 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:45.420+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:45.526 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:45.524+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:45.553 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:45.553 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:45.553 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:45.553 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:45.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:45.608 INFO:teuthology.orchestra.run.smithi170.stdout:5835 write 0xb1fa600 thru 0xb20a234 (0xfc35 bytes) 2024-09-21T23:17:45.611 INFO:teuthology.orchestra.run.smithi170.stdout:5836 read 0x8f92600 thru 0x8f978c0 (0x52c1 bytes) 2024-09-21T23:17:45.611 INFO:teuthology.orchestra.run.smithi170.stdout:5840 write 0x110b000 thru 0x1118811 (0xd812 bytes) 2024-09-21T23:17:45.612 INFO:teuthology.orchestra.run.smithi170.stdout:5841 write 0x1545c00 thru 0x1550012 (0xa413 bytes) 2024-09-21T23:17:45.615 INFO:teuthology.orchestra.run.smithi170.stdout:5842 write 0x6c31e00 thru 0x6c364e6 (0x46e7 bytes) 2024-09-21T23:17:45.615 INFO:teuthology.orchestra.run.smithi170.stdout:5846 write 0x77b1200 thru 0x77b80da (0x6edb bytes) 2024-09-21T23:17:45.615 INFO:teuthology.orchestra.run.smithi170.stdout:5853 read 0x5cc4a00 thru 0x5cc6184 (0x1785 bytes) 2024-09-21T23:17:45.615 INFO:teuthology.orchestra.run.smithi170.stdout:5857 punch from 0x5197600 to 0x51a4e00, (0xd800 bytes) 2024-09-21T23:17:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:5858 read 0x3243000 thru 0x324aec9 (0x7eca bytes) 2024-09-21T23:17:45.626 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:45.622+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:45.668 INFO:teuthology.orchestra.run.smithi170.stdout:5860 trunc from 0xb20a400 to 0x3391a00 2024-09-21T23:17:45.694 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:45.694 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:45.695 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:45.695 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:45.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:45.759 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:45.758+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:45.788 INFO:teuthology.orchestra.run.smithi170.stdout:5862 write 0xd9ba800 thru 0xd9bde2b (0x362c bytes) 2024-09-21T23:17:45.789 INFO:teuthology.orchestra.run.smithi170.stdout:5863 punch from 0x9831600 to 0x9837800, (0x6200 bytes) 2024-09-21T23:17:45.789 INFO:teuthology.orchestra.run.smithi170.stdout:5864 punch from 0x98e4200 to 0x98f1c00, (0xda00 bytes) 2024-09-21T23:17:45.805 INFO:teuthology.orchestra.run.smithi170.stdout:5866 trunc from 0xd9be000 to 0x5189e00 2024-09-21T23:17:45.828 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:45.824+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:45.870 INFO:teuthology.orchestra.run.smithi170.stdout:5868 write 0x7ffc000 thru 0x8004582 (0x8583 bytes) 2024-09-21T23:17:45.871 INFO:teuthology.orchestra.run.smithi170.stdout:5871 punch from 0x16a6600 to 0x16b5a00, (0xf400 bytes) 2024-09-21T23:17:45.874 INFO:teuthology.orchestra.run.smithi170.stdout:5872 write 0x3f4a000 thru 0x3f4fb95 (0x5b96 bytes) 2024-09-21T23:17:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:5873 write 0x18f1e00 thru 0x18f738a (0x558b bytes) 2024-09-21T23:17:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:5874 read 0x4c2bc00 thru 0x4c316e3 (0x5ae4 bytes) 2024-09-21T23:17:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:5876 clone 40 order 19 su 262144 sc 8 2024-09-21T23:17:45.903 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97547288800001, 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97445204100001, 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.914 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.974392368, 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:45.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:45.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:45.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.974302392, 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:45.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:45.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.974252482, 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.974196342, 2024-09-21T23:17:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:45.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.934 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.974142559, 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:45.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.939 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97411028299999, 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.940 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:45.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:45.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97408000300001, 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:45.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:45.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:45.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.974050347, 2024-09-21T23:17:45.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:45.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:45.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.955 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97402038000001, 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:45.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97394255099999, 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.961 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.973894844, 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.966 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:45.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97386464799999, 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:45.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:45.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:45.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97383473100001, 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97380493099999, 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97377499800001, 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.985 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:45.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:45.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.989 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97371003500001, 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:45.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:45.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.973679559, 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:45.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:45.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:45.999 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.97365039499999, 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:46.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:46.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:46.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:46.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:46.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:46.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:46.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:46.005 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:46.006 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:46.007 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:46.004+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:46.107 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:46.104+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:46.136 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:46.136 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:46.136 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:46.137 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:46.152 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:46.208 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:46.204+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:46.310 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:46.306+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:46.320 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:46.320 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:46.320 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:46.320 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:46.320 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:46.335 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:46.357 INFO:teuthology.orchestra.run.smithi170.stdout:leaving image image_client.0-clone39 intact 2024-09-21T23:17:46.410 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:46.406+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:46.510 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:46.506+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:46.515 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:46.515 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 180.29282317299999, 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.78041789, 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.522 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:46.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.78035298699999, 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.780206548, 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.538 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.78013743299999, 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.779525108, 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.779320703, 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.779277917, 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38831962899999, 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.561 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:46.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38823311900001, 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38815622999999, 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:46.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:46.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:46.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:46.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38795139000001, 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:46.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.387863108, 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:46.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38773227999999, 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:46.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:46.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38768662699999, 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.597 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:46.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.597 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:46.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:46.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.3876482, 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.602 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:46.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:46.607 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38751873800001, 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.613 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.387497096, 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:46.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:46.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:46.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38747705200001, 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:46.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.625 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 179.38739305999999, 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:46.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:46.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:46.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:46.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:46.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:46.631 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:46.632 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:46.633 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:46.630+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:46.733 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:46.730+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:46.759 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:46.759 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:46.760 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:46.760 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:46.774 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:46.834 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:46.830+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:46.901 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:46.901 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:46.902 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:46.902 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:46.915 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:46.962 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:46.960+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:46.974 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 132.078425388, 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:47.036 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:47.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 132.07815840399999, 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.042 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:47.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:47.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465829858999996, 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:47.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:47.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465787160000005, 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:47.052 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:47.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465748540000007, 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:47.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:47.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465716667999999, 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.061 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:47.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465683444000007, 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:47.065 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:47.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465648368999993, 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:47.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:47.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:47.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465603411000004, 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.074 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:47.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:47.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.465151344000006, 2024-09-21T23:17:47.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.464436864999996, 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:47.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.464395795000001, 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.087 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.091 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.463938677000002, 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.095 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.463701998000005, 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:47.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:47.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.463286417000006, 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.100 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:47.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:47.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:47.104 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.462608756999998, 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.462117468000002, 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.109 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 90.461860569999999, 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:47.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.752669470999997, 2024-09-21T23:17:47.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:47.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.749790769999997, 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:17:47.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.127 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:47.128 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:47.032+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:47.129 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:47.170 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:47.168+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:47.188 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:47.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:17:47.237 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:47.234+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5971.0:1057 3.1c 3:3a46deb0:::rbd_data.10f1e1bbb8ea.0000000000000002:head [sparse-read 3342336~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:47.312058+0000", 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00042125499999999999, 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00048938200000000001, 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5971", 2024-09-21T23:17:47.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/521448765", 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1057 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312058+0000", 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312058+0000", 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312059+0000", 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3399999999999998e-07 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312063+0000", 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5499999999999996e-06 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312074+0000", 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0845999999999999e-05 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312082+0000", 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7060000000000004e-06 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312182+0000", 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9826000000000003e-05 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312300+0000", 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000118199 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5971.0:1059 3.10 3.856e58f0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:47.312582+0000", 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.999898, 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014438600000000001, 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5971", 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/521448765", 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1059 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:47.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312582+0000", 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312582+0000", 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312584+0000", 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0530000000000001e-06 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312596+0000", 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.198e-05 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312618+0000", 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.232e-05 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312638+0000", 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0080000000000001e-05 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312693+0000", 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5445999999999997e-05 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:47.312739+0000", 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5643999999999999e-05 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 2 2024-09-21T23:17:47.320 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:47.329 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:47.356 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:47.373 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:47.372+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:47.378 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:47.378 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:47.398 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:47.398 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:47.419 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:47.439 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:47.438+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:47.442 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:47.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:47.462 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:47.462 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:47.462 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:47.462 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:47.462 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:47.467 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:47.467 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:47.477 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:47.540 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:47.538+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:47.640 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:47.636+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:47.647 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:47.647 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:47.662 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:47.741 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:47.738+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:47.802 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:47.802 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:47.817 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:47.863 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:47.862+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:47.943 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:47.940+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:47.994 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:47.994 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:48.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:48.044 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:48.040+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:48.061 INFO:teuthology.orchestra.run.smithi170.stdout:5877 read 0x3b14e00 thru 0x3b225ab (0xd7ac bytes) 2024-09-21T23:17:48.063 INFO:teuthology.orchestra.run.smithi170.stdout:5882 read 0x2741000 thru 0x2742cf0 (0x1cf1 bytes) 2024-09-21T23:17:48.064 INFO:teuthology.orchestra.run.smithi170.stdout:5883 read 0x6228400 thru 0x622d08d (0x4c8e bytes) 2024-09-21T23:17:48.065 INFO:teuthology.orchestra.run.smithi170.stdout:5884 read 0x6acf000 thru 0x6ad8497 (0x9498 bytes) 2024-09-21T23:17:48.066 INFO:teuthology.orchestra.run.smithi170.stdout:5885 read 0x4018a00 thru 0x40272a1 (0xe8a2 bytes) 2024-09-21T23:17:48.068 INFO:teuthology.orchestra.run.smithi170.stdout:5891 write 0x46eb800 thru 0x46f9419 (0xdc1a bytes) 2024-09-21T23:17:48.069 INFO:teuthology.orchestra.run.smithi170.stdout:5892 write 0x2c61400 thru 0x2c69bd6 (0x87d7 bytes) 2024-09-21T23:17:48.071 INFO:teuthology.orchestra.run.smithi170.stdout:5894 write 0x756a600 thru 0x7571552 (0x6f53 bytes) 2024-09-21T23:17:48.107 INFO:teuthology.orchestra.run.smithi170.stdout:5896 trunc from 0x8004600 to 0x173b600 2024-09-21T23:17:48.139 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:48.139 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:48.145 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:48.140+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:48.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:48.195 INFO:teuthology.orchestra.run.smithi170.stdout:5897 trunc from 0x173b600 to 0x4453a00 2024-09-21T23:17:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:17:48.246 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:48.242+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:48.282 INFO:teuthology.orchestra.run.smithi170.stdout:5899 write 0x9b1b400 thru 0x9b20a29 (0x562a bytes) 2024-09-21T23:17:48.289 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:48.289 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:48.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:48.347 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:48.344+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:48.368 INFO:teuthology.orchestra.run.smithi170.stdout:5901 write 0xdd6a200 thru 0xdd75feb (0xbdec bytes) 2024-09-21T23:17:48.390 INFO:teuthology.orchestra.run.smithi170.stdout:5904 trunc from 0xdd76000 to 0x5cc6600 2024-09-21T23:17:48.447 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:48.444+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:48.454 INFO:teuthology.orchestra.run.smithi170.stdout:5905 trunc from 0x5cc6600 to 0x1377000 2024-09-21T23:17:48.473 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:48.473 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:48.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:48.548 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:48.544+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:48.554 INFO:teuthology.orchestra.run.smithi170.stdout:5906 write 0x9fc3000 thru 0x9fca81c (0x781d bytes) 2024-09-21T23:17:48.556 INFO:teuthology.orchestra.run.smithi170.stdout:5907 punch from 0x96f4400 to 0x9700600, (0xc200 bytes) 2024-09-21T23:17:48.557 INFO:teuthology.orchestra.run.smithi170.stdout:5908 write 0x416d000 thru 0x417529f (0x82a0 bytes) 2024-09-21T23:17:48.557 INFO:teuthology.orchestra.run.smithi170.stdout:5910 punch from 0x333aa00 to 0x334a400, (0xfa00 bytes) 2024-09-21T23:17:48.558 INFO:teuthology.orchestra.run.smithi170.stdout:5912 punch from 0x850b400 to 0x8516200, (0xae00 bytes) 2024-09-21T23:17:48.558 INFO:teuthology.orchestra.run.smithi170.stdout:5914 read 0x2d8000 thru 0x2e5174 (0xd175 bytes) 2024-09-21T23:17:48.559 INFO:teuthology.orchestra.run.smithi170.stdout:skipping zero length punch hole 2024-09-21T23:17:48.559 INFO:teuthology.orchestra.run.smithi170.stdout:5916 punch from 0x9948000 to 0x9956e00, (0xee00 bytes) 2024-09-21T23:17:48.560 INFO:teuthology.orchestra.run.smithi170.stdout:5921 punch from 0x3c12400 to 0x3c20600, (0xe200 bytes) 2024-09-21T23:17:48.560 INFO:teuthology.orchestra.run.smithi170.stdout:5922 read 0x728c600 thru 0x7293145 (0x6b46 bytes) 2024-09-21T23:17:48.561 INFO:teuthology.orchestra.run.smithi170.stdout:5923 write 0x324f200 thru 0x325129d (0x209e bytes) 2024-09-21T23:17:48.561 INFO:teuthology.orchestra.run.smithi170.stdout:5924 read 0x1946000 thru 0x1946174 (0x175 bytes) 2024-09-21T23:17:48.561 INFO:teuthology.orchestra.run.smithi170.stdout:5926 read 0x613cc00 thru 0x613e277 (0x1678 bytes) 2024-09-21T23:17:48.561 INFO:teuthology.orchestra.run.smithi170.stdout:5928 write 0x37e9400 thru 0x37f0b0f (0x7710 bytes) 2024-09-21T23:17:48.561 INFO:teuthology.orchestra.run.smithi170.stdout:5929 read 0x5477200 thru 0x548097a (0x977b bytes) 2024-09-21T23:17:48.561 INFO:teuthology.orchestra.run.smithi170.stdout:5930 punch from 0x4630000 to 0x4634600, (0x4600 bytes) 2024-09-21T23:17:48.613 INFO:teuthology.orchestra.run.smithi170.stdout:5932 write 0xad25000 thru 0xad33c26 (0xec27 bytes) 2024-09-21T23:17:48.616 INFO:teuthology.orchestra.run.smithi170.stdout:5933 trunc from 0xad33e00 to 0xba14800 2024-09-21T23:17:48.620 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:48.620 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:48.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:48.652 INFO:teuthology.orchestra.run.smithi170.stdout:5934 read 0x5000800 thru 0x500b39a (0xab9b bytes) 2024-09-21T23:17:48.654 INFO:teuthology.orchestra.run.smithi170.stdout:5935 read 0x13f400 thru 0x14c8e7 (0xd4e8 bytes) 2024-09-21T23:17:48.679 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:48.678+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:48.681 INFO:teuthology.orchestra.run.smithi170.stdout:5936 trunc from 0xba14800 to 0x1b9400 2024-09-21T23:17:48.748 INFO:teuthology.orchestra.run.smithi170.stdout:5937 read 0xc400 thru 0x1c305 (0xff06 bytes) 2024-09-21T23:17:48.749 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:48.746+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:48.753 INFO:teuthology.orchestra.run.smithi170.stdout:5938 read 0x16aa00 thru 0x1731eb (0x87ec bytes) 2024-09-21T23:17:48.754 INFO:teuthology.orchestra.run.smithi170.stdout:5939 read 0x91c00 thru 0xa17a8 (0xfba9 bytes) 2024-09-21T23:17:48.755 INFO:teuthology.orchestra.run.smithi170.stdout:5941 read 0x90600 thru 0x96dc0 (0x67c1 bytes) 2024-09-21T23:17:48.756 INFO:teuthology.orchestra.run.smithi170.stdout:5942 read 0x35800 thru 0x4392a (0xe12b bytes) 2024-09-21T23:17:48.761 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:48.761 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:48.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:48.789 INFO:teuthology.orchestra.run.smithi170.stdout:5943 write 0x5538200 thru 0x5547bfd (0xf9fe bytes) 2024-09-21T23:17:48.791 INFO:teuthology.orchestra.run.smithi170.stdout:5945 read 0x1293000 thru 0x129d589 (0xa58a bytes) 2024-09-21T23:17:48.791 INFO:teuthology.orchestra.run.smithi170.stdout:5946 write 0x34b2200 thru 0x34bbf28 (0x9d29 bytes) 2024-09-21T23:17:48.841 INFO:teuthology.orchestra.run.smithi170.stdout:5947 write 0xb49ac00 thru 0xb4a2dd6 (0x81d7 bytes) 2024-09-21T23:17:48.845 INFO:teuthology.orchestra.run.smithi170.stdout:5951 punch from 0xa690800 to 0xa697200, (0x6a00 bytes) 2024-09-21T23:17:48.846 INFO:teuthology.orchestra.run.smithi170.stdout:5952 write 0x7a83200 thru 0x7a8861c (0x541d bytes) 2024-09-21T23:17:48.863 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:48.862+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:48.865 INFO:teuthology.orchestra.run.smithi170.stdout:5953 trunc from 0xb4a2e00 to 0xa12a00 2024-09-21T23:17:48.948 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:48.949 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:48.951 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:48.950+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:48.963 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:48.965 INFO:teuthology.orchestra.run.smithi170.stdout:5955 write 0xcb6dc00 thru 0xcb796b9 (0xbaba bytes) 2024-09-21T23:17:48.966 INFO:teuthology.orchestra.run.smithi170.stdout:5956 read 0x4692800 thru 0x46933d5 (0xbd6 bytes) 2024-09-21T23:17:48.967 INFO:teuthology.orchestra.run.smithi170.stdout:5957 read 0x9dd600 thru 0x9e6207 (0x8c08 bytes) 2024-09-21T23:17:48.967 INFO:teuthology.orchestra.run.smithi170.stdout:5962 punch from 0x1823a00 to 0x1826e00, (0x3400 bytes) 2024-09-21T23:17:48.967 INFO:teuthology.orchestra.run.smithi170.stdout:5963 read 0x8667600 thru 0x8673068 (0xba69 bytes) 2024-09-21T23:17:48.968 INFO:teuthology.orchestra.run.smithi170.stdout:5964 write 0x493e800 thru 0x494434a (0x5b4b bytes) 2024-09-21T23:17:48.968 INFO:teuthology.orchestra.run.smithi170.stdout:5966 read 0xc2f6000 thru 0xc2fcc0d (0x6c0e bytes) 2024-09-21T23:17:48.968 INFO:teuthology.orchestra.run.smithi170.stdout:5968 read 0x7ddba00 thru 0x7deb94e (0xff4f bytes) 2024-09-21T23:17:48.968 INFO:teuthology.orchestra.run.smithi170.stdout:5971 read 0xb223000 thru 0xb226381 (0x3382 bytes) 2024-09-21T23:17:49.007 INFO:teuthology.orchestra.run.smithi170.stdout:5975 write 0xcd04400 thru 0xcd08c0d (0x480e bytes) 2024-09-21T23:17:49.008 INFO:teuthology.orchestra.run.smithi170.stdout:5976 read 0x3d6800 thru 0x3ddb3a (0x733b bytes) 2024-09-21T23:17:49.015 INFO:teuthology.orchestra.run.smithi170.stdout:5977 trunc from 0xcd08e00 to 0xa815a00 2024-09-21T23:17:49.052 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:49.048+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:49.055 INFO:teuthology.orchestra.run.smithi170.stdout:5981 write 0x77ee200 thru 0x77f9055 (0xae56 bytes) 2024-09-21T23:17:49.057 INFO:teuthology.orchestra.run.smithi170.stdout:5982 write 0xa0e3c00 thru 0xa0f1b30 (0xdf31 bytes) 2024-09-21T23:17:49.057 INFO:teuthology.orchestra.run.smithi170.stdout:5983 read 0x1d64a00 thru 0x1d64cb9 (0x2ba bytes) 2024-09-21T23:17:49.057 INFO:teuthology.orchestra.run.smithi170.stdout:5984 punch from 0x9a10400 to 0x9a16400, (0x6000 bytes) 2024-09-21T23:17:49.093 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:49.093 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:49.093 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T23:17:49.093 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-21T23:17:49.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T23:17:49.094 INFO:teuthology.orchestra.run.smithi170.stdout:5985 write 0xac29400 thru 0xac2f6d4 (0x62d5 bytes) 2024-09-21T23:17:49.131 INFO:teuthology.orchestra.run.smithi170.stdout:5986 write 0xb1d5e00 thru 0xb1e19af (0xbbb0 bytes) 2024-09-21T23:17:49.133 INFO:teuthology.orchestra.run.smithi170.stdout:5988 write 0x6667400 thru 0x6669d24 (0x2925 bytes) 2024-09-21T23:17:49.133 INFO:teuthology.orchestra.run.smithi170.stdout:5989 read 0x47e8800 thru 0x47ec9e2 (0x41e3 bytes) 2024-09-21T23:17:49.133 INFO:teuthology.orchestra.run.smithi170.stdout:5991 write 0xa684c00 thru 0xa68a840 (0x5c41 bytes) 2024-09-21T23:17:49.136 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:49.136 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:49.152 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:49.150+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:49.153 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:49.176 INFO:teuthology.orchestra.run.smithi170.stdout:5993 write 0xd9d2000 thru 0xd9dd654 (0xb655 bytes) 2024-09-21T23:17:49.176 INFO:teuthology.orchestra.run.smithi170.stdout:5995 read 0xbf15400 thru 0xbf2491e (0xf51f bytes) 2024-09-21T23:17:49.211 INFO:teuthology.orchestra.run.smithi170.stdout:5996 trunc from 0xd9dd800 to 0x110800 2024-09-21T23:17:49.253 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:49.248+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:49.283 INFO:teuthology.orchestra.run.smithi170.stdout:5997 trunc from 0x110800 to 0xbba3600 2024-09-21T23:17:49.293 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:49.293 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:49.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:49.309 INFO:teuthology.orchestra.run.smithi170.stdout:5999 write 0x5311a00 thru 0x53213b5 (0xf9b6 bytes) 2024-09-21T23:17:49.309 INFO:teuthology.orchestra.run.smithi170.stdout:6000 write 0x5c1400 thru 0x5c1f04 (0xb05 bytes) 2024-09-21T23:17:49.353 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:49.350+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:49.454 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:49.450+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:49.470 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:49.470 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:49.484 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:49.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:49.555 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:49.554+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:49.604 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:49.605 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:49.618 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:49.640 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:49.640 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:49.655 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:49.654+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:49.656 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:49.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:49.673 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:49.673 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:49.691 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:49.691 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:49.711 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:49.711 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:49.730 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:49.731 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:49.737 INFO:teuthology.orchestra.run.smithi170.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5995.0:62 3.a 3:50f02fd6:::rbd_data.10f168a2bf1f.000000000000001f:head [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:49.740606+0000", 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.000108779, 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015785500000000001, 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5995", 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3099152725", 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 62 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740606+0000", 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740606+0000", 2024-09-21T23:17:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740608+0000", 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3200000000000001e-06 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740614+0000", 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5869999999999998e-06 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740634+0000", 2024-09-21T23:17:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9712999999999999e-05 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740644+0000", 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0010000000000001e-05 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740696+0000", 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2453000000000001e-05 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:49.740753+0000", 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6375000000000001e-05 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:17:49.745 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:49.757 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:49.791 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:49.790+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:49.857 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:49.854+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:49.882 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:49.882 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:49.882 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:49.882 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:49.882 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:49.898 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:49.958 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:49.956+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 135.068052421, 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:50.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:17:50.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 135.067785437, 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.030 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:17:50.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:17:50.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:17:50.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:17:50.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.455456892000001, 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.035 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:50.039 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.455414192999996, 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:50.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:17:50.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:17:50.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.455375572999998, 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:17:50.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:17:50.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.455343701000004, 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:17:50.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:17:50.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.455310476999998, 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.052 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:17:50.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:17:50.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:17:50.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.455275401999998, 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.056 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:17:50.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:17:50.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:17:50.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.455230443999994, 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:17:50.060 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.064 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.454778376999997, 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:17:50.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.454063898000001, 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:17:50.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:17:50.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.072 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.454022828000006, 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:50.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:17:50.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:17:50.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.076 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.453565710000007, 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:50.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.453329030999996, 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.452913449999997, 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.088 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.452235790000003, 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.089 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:17:50.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:17:50.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.451744500999993, 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:17:50.093 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:17:50.097 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 93.451487603000004, 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:17:50.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 36.742296504000002, 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:17:50.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:17:50.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:17:50.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 36.739417803000002, 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:50.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:17:50.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:17:50.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.111 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:50.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:50.113 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:50.110+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:50.213 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:50.212+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:50.224 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T23:17:50.242 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:17:50.297 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:50.298 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:50.298 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:50.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:50.357 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:50.356+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:50.414 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:50.410+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:50.485 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:50.486 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:50.486 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:50.486 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:50.486 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:50.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:50.516 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:50.514+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:50.617 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:50.612+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.372803249999997, 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.626 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:17:50.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:17:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.347754291000001, 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:50.632 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:17:50.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:17:50.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:17:50.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:50.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.314321120999999, 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:17:50.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.641 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1238 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3435706609999993, 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054982073999999999, 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1238 2024-09-21T23:17:50.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280879+0000", 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8700000000000005e-07 2024-09-21T23:17:50.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280883+0000", 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.912e-06 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280894+0000", 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1275e-05 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280904+0000", 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.73e-06 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335407+0000", 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054502816000000003 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335499+0000", 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2589000000000004e-05 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335860+0000", 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036096499999999998 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.645 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1341 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3401391530000009, 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055656854999999998, 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1341 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.646 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284311+0000", 2024-09-21T23:17:50.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.06e-06 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284314+0000", 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.72e-06 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284328+0000", 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3380000000000001e-05 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284338+0000", 2024-09-21T23:17:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0052e-05 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338191+0000", 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053852786999999999 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338273+0000", 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2021000000000006e-05 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339967+0000", 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016938350000000001 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1342 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 966656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3400991730000005, 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056087662000000003, 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.650 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1342 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:17:50.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284354+0000", 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.569e-06 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284364+0000", 2024-09-21T23:17:50.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0710000000000001e-05 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284373+0000", 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7819999999999999e-06 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340014+0000", 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055641057000000001 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340135+0000", 2024-09-21T23:17:50.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120698 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340437+0000", 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030229200000000001 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1343 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3400611900000001, 2024-09-21T23:17:50.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055970117, 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1343 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9200000000000001e-07 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284392+0000", 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8240000000000001e-06 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284404+0000", 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1584e-05 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284415+0000", 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1412e-05 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339993+0000", 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055577519999999998 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340056+0000", 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3127999999999996e-05 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340358+0000", 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000302057 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1344 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.658 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3400218259999992, 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056410920000000003, 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1344 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:17:50.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284428+0000", 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284432+0000", 2024-09-21T23:17:50.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9060000000000004e-06 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284443+0000", 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.134e-05 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284452+0000", 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0340000000000002e-06 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340391+0000", 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055938437000000001 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340464+0000", 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3744000000000004e-05 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340838+0000", 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000373595 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1345 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3399862260000006, 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056714902999999997, 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1345 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.663 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284464+0000", 2024-09-21T23:17:50.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284467+0000", 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.63e-06 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284479+0000", 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.17e-05 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284488+0000", 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1870000000000003e-06 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340886+0000", 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056398177000000001 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340951+0000", 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4263000000000005e-05 2024-09-21T23:17:50.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341178+0000", 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022679199999999999 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1346 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3399197530000002, 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057046119999999999, 2024-09-21T23:17:50.667 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1346 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284530+0000", 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284535+0000", 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7960000000000003e-06 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284548+0000", 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3002e-05 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284558+0000", 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8640000000000008e-06 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341212+0000", 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056654228000000001 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341281+0000", 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9061999999999997e-05 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341575+0000", 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000294152 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.671 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1419 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 32768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3390721929999998, 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056364545000000002, 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1419 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:17:50.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285381+0000", 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:17:50.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285391+0000", 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.03e-05 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285400+0000", 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1959999999999997e-06 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341615+0000", 2024-09-21T23:17:50.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056215380000000002 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341674+0000", 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8403000000000003e-05 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341741+0000", 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7352000000000002e-05 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1420 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3390361810000009, 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056531351000000001, 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1420 2024-09-21T23:17:50.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:17:50.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3799999999999997e-07 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285417+0000", 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0459999999999999e-06 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285429+0000", 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1402e-05 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285438+0000", 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2050000000000008e-06 2024-09-21T23:17:50.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341791+0000", 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056353542999999999 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341861+0000", 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9338000000000006e-05 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341944+0000", 2024-09-21T23:17:50.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3178999999999998e-05 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1421 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3389964820000007, 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056621478000000003, 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1421 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285453+0000", 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.066e-06 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285457+0000", 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285468+0000", 2024-09-21T23:17:50.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0974e-05 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285477+0000", 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0459999999999994e-06 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341982+0000", 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056504285000000001 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342029+0000", 2024-09-21T23:17:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7528999999999998e-05 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342074+0000", 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4647999999999997e-05 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1424 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3364903580000007, 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054910207000000003, 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.684 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1424 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287959+0000", 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0139999999999999e-06 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287965+0000", 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287989+0000", 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4511e-05 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288003+0000", 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3318e-05 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342685+0000", 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054682236000000002 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342764+0000", 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9122999999999994e-05 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342869+0000", 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010467399999999999 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1448 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:17:50.688 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3359641569999994, 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055210594000000002, 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1448 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288486+0000", 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4200000000000004e-07 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288490+0000", 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8979999999999998e-06 2024-09-21T23:17:50.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288501+0000", 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288509+0000", 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7499999999999992e-06 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343373+0000", 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05486365 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343441+0000", 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8119000000000003e-05 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343695+0000", 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025414099999999999 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1449 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:17:50.692 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3359281250000006, 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055644921999999999, 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1449 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4600000000000005e-07 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288525+0000", 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9519999999999996e-06 2024-09-21T23:17:50.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288536+0000", 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0862e-05 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288544+0000", 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.276e-06 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343752+0000", 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055208113000000003 2024-09-21T23:17:50.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343837+0000", 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4466000000000001e-05 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344166+0000", 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032870700000000002 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1450 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.696 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.335892973, 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055497638000000002, 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1450 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288557+0000", 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288562+0000", 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7860000000000002e-06 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288574+0000", 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1637999999999999e-05 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288583+0000", 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8240000000000006e-06 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343733+0000", 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055150504000000003 2024-09-21T23:17:50.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343785+0000", 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1805999999999998e-05 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344054+0000", 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026842000000000001 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.700 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1451 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3358545450000001, 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055892521000000001, 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1451 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:17:50.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288595+0000", 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288599+0000", 2024-09-21T23:17:50.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1999999999999996e-06 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288611+0000", 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1506e-05 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288619+0000", 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7790000000000001e-06 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344089+0000", 2024-09-21T23:17:50.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055469472999999998 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344170+0000", 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1287999999999998e-05 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344487+0000", 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031666499999999999 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1452 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3358183639999996, 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056290277999999999, 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1452 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288631+0000", 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288635+0000", 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0339999999999998e-06 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288646+0000", 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1107999999999999e-05 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288656+0000", 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3319999999999998e-06 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344531+0000", 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055875073999999997 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344606+0000", 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4838000000000004e-05 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344921+0000", 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031523599999999999 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1453 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.3357825079999994, 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056627027000000003, 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1453 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:50.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288667+0000", 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288671+0000", 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.293991+0000", 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053204910000000001 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.294012+0000", 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1019999999999999e-05 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344962+0000", 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050949529 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345048+0000", 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6263999999999999e-05 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345293+0000", 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000245135 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.712 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:50.713 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:50.713 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:50.714 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:50.717 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:50.714+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:50.817 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:50.816+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:50.849 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:50.849 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:50.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:50.918 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:50.914+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:50.994 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:50.995 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:50.995 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:51.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:51.019 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:51.016+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:51.120 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:51.118+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.02148265, 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.956227892, 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 99.558885759999995, 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 99.555518070999995, 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:51.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 99.551408491000004, 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 99.547615570000005, 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483699692000002, 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.298 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483672890999998, 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:51.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:51.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:51.303 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483646726000003, 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:51.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:51.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483618276000001, 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.308 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483527568, 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483495989000005, 2024-09-21T23:17:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483458299000006, 2024-09-21T23:17:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483427909, 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483384491999999, 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.483349867000001, 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.480096367000002, 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.480061785000004, 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.264317386000002, 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.655342190000001, 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:17:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:17:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.359 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:51.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:17:51.362 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:51.356+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:51.461 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:51.458+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:51.551 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:51.551 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:51.551 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:51.551 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:51.562 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:51.560+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:51.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:51.663 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:51.660+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:51.735 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:51.735 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:51.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:51.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:51.763 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:51.760+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:51.864 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:51.862+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.122230576000007, 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.122185318999996, 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.122135697000004, 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.122106001999995, 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.122073073999999, 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:51.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.122038778000004, 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:51.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.122014272000001, 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.910 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121986281999995, 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.915 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:51.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:51.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121963253999994, 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:51.919 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:51.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:51.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121941743999997, 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:51.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121906202999995, 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:51.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:51.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121878566000007, 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:51.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121843401999996, 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:51.938 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:51.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:51.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:51.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121808371, 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:51.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:51.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121772985999996, 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121732804000004, 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:51.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121695676000002, 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:51.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121655601000001, 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121631057000002, 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:51.966 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:51.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:51.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.121593271999998, 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:51.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:51.975 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:51.976 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:51.976 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:51.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:51.977 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:51.978 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:51.978 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:51.978 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:51.978 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:51.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:51.979 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:51.979 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:51.980 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:51.972+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:51.988 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:51.988 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:52.077 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:52.076+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:52.107 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:52.107 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:52.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:52.179 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:52.176+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:52.256 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:52.256 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:52.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:52.280 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:52.276+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:52.381 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:52.376+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:52.415 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:52.415 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:52.430 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:52.481 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:52.480+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:52.565 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:52.565 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:52.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:52.582 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:52.582+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:52.683 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:52.680+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:52.743 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:52.743 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:52.756 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:52.784 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:52.786+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:52.885 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:52.882+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:52.926 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:52.926 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:52.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:52.986 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:52.984+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:53.055 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:53.055 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:53.069 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:53.114 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:53.112+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:53.188 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:53.186+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:53.238 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:53.238 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:53.253 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:53.290 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:53.286+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:53.362 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:17:53.373 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:53.374 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:53.388 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:53.408 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:53.406+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:53.490 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:53.488+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:53.522 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:53.523 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:53.537 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:53.625 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:53.624+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:53.692 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:53.690+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:53.695 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:53.695 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:53.709 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:53.793 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:53.794+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:53.877 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:53.877 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:53.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:53.893 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:53.892+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:53.995 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:53.994+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:54.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:54.024 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:54.024 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:54.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:54.096 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:54.092+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:54.150 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:54.150 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:54.169 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:54.169 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:54.187 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:54.188 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:54.196 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:54.194+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:54.209 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:54.209 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:54.209 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:54.210 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:54.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:54.230 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:54.230 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:54.251 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:54.251 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:54.252 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:17:54.331 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:54.330+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 112.20465453, 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:17:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:17:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:17:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.139399772, 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:17:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:17:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 102.74205764, 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:17:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:17:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 102.738689951, 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 102.73458037100001, 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:17:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:17:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 102.73078744999999, 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:17:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.66687157200001, 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:17:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:17:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.666844771, 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.66681860600001, 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:17:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:17:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:17:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.666790156, 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:17:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.666699448, 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:17:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:17:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.66666786899999, 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:17:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.66663017899999, 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.666599789, 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:17:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:17:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.666556372, 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.666521747, 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:17:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.663268247, 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:17:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:17:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:17:54.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.66323366500001, 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:17:54.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.447489266000005, 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:17:54.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.838514069999999, 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:17:54.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:17:54.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.462 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:54.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:54.464 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:54.462+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:54.565 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:54.560+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:54.646 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:54.646 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:54.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:54.665 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:54.660+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:54.766 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:54.762+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:54.785 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:54.785 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:54.785 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:54.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:54.891 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:54.890+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:54.968 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:54.966+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209361186999999, 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:17:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209315930000002, 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:17:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209266307999997, 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:17:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:17:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:17:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209236613000002, 2024-09-21T23:17:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209203685000006, 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:17:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:17:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209169388999996, 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:17:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209144882999993, 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:17:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209116893000001, 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:17:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:17:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209093865, 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209072355000004, 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:17:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209036814000001, 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:17:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.209009176999999, 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:17:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:17:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208974013000002, 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:17:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:17:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208938982000006, 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208903597000003, 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.044 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208863414999996, 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:17:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:17:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208826286999994, 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:17:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:17:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208786212000007, 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:17:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:17:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208761667999994, 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.208723883000005, 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:17:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.069 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:55.071 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:55.071 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:55.068+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:55.171 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:55.168+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:55.246 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:55.246 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:17:55.246 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:55.249 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:17:55.249 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:17:55.249 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T23:17:55.249 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:17:55.250 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:17:55.250 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T23:17:55.261 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:55.307 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:55.306+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:55.373 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:55.370+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:55.391 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:55.391 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "" 2024-09-21T23:17:55.391 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:55.392 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:55.392 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:17:55.392 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:17:55.392 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:17:55.392 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:55.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:55.499 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:55.498+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:55.574 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:55.572+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65499504799999, 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65397420100001, 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.653914528, 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:55.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.653824552, 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:55.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:55.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:55.606 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.653774642, 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:55.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:55.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.653718502, 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:55.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65366471900001, 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:55.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:55.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:55.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65363244299999, 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:55.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:55.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65360216299999, 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:55.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:55.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65357250700001, 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.635 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:55.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65354253999999, 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:55.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.653464711, 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.647 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:55.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.653417004, 2024-09-21T23:17:55.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:55.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:55.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:55.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65338680799999, 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:55.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65335689099999, 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65332709099999, 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:55.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65329715799999, 2024-09-21T23:17:55.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:55.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:55.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65323219499999, 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:55.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:55.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.65320171900001, 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:55.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:55.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.653172555, 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:55.692 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:55.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:55.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:55.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:55.698 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:55.699 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:55.699 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:55.696+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:55.800 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:55.802+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:55.829 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:55.830 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:55.844 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:55.901 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:55.896+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:55.967 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:55.967 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:55.967 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:55.967 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:55.968 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:55.981 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:56.002 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:56.002+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:56.102 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:56.100+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:56.149 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:56.149 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:56.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:56.149 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.92683758800001, 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:56.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:56.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:56.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.156 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.41443230499999, 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:56.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.41436740200001, 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:56.162 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:56.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:56.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.41422096299999, 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.168 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:56.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:56.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.41415184799999, 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:56.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:56.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.179 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.413539523, 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.180 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:56.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:56.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.41333511799999, 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.186 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:56.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:56.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:56.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.413292332, 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:56.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:56.197 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.02233404399999, 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.198 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:56.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.022247534, 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.204 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:56.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:56.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.02217064499999, 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:56.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:56.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.02196580500001, 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:56.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.221 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.021877523, 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:56.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.02174669499999, 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:56.227 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:56.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:56.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.02170104199999, 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.233 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:56.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.239 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.021662615, 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:56.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:56.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:56.245 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.02153315300001, 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:56.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:56.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.021511511, 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.251 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:56.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:56.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.021491467, 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.257 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:56.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 189.02140747499999, 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:56.263 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:56.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:56.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:56.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:56.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.269 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:56.269 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:56.269 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:56.270 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:56.271 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:56.268+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:56.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:17:56.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:17:56.375 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:56.374+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:56.443 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:56.461 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:56.464 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:56.464 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:56.464 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:56.464 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:56.471 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:56.468+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:56.479 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:56.479 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:56.480 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:56.500 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:56.500 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:56.522 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:56.522 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:56.542 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:56.542 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:56.603 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:56.602+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:56.651 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:56.651 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:56.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:56.710 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:56.708+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:56.774 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:56.770+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:56.788 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:56.788 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:56.803 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:17:56.889 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:56.888+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:56.963 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:56.963 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:56.975 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:56.974+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:56.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:17:57.076 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:57.074+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:57.150 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:57.150 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:57.165 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:17:57.176 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:57.174+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:57.277 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:57.276+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:57.281 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:57.282 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:57.293 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:17:57.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:17:57.378 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:57.374+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:57.434 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:57.434 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:57.450 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:17:57.491 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:57.490+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:57.579 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:57.578+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:57.618 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:57.618 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:57.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:17:57.723 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:57.722+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:57.782 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:57.780+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:57.804 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:57.804 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:57.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:17:57.883 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:57.880+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:57.983 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:57.980+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:57.984 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:57.984 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:57.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:17:58.085 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:17:58.082+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:17:58.123 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:58.124 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:58.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:17:58.186 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:58.184+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:58.287 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:58.286+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:58.307 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:58.307 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:58.324 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:17:58.388 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:58.384+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:58.490 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:58.486+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:58.497 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:58.497 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:58.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:17:58.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:17:58.599 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:58.598+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:58.685 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:17:58.686 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:17:58.691 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:58.690+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:58.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:17:58.744 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:17:58.744 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:17:58.761 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:17:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:17:58.779 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:17:58.779 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:17:58.791 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:58.788+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:58.800 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:17:58.800 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:17:58.821 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:17:58.821 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:17:58.843 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:17:58.843 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93476735199999, 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:17:58.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:17:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93374650499999, 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:17:58.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:17:58.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93368683200001, 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:17:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93359685600001, 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:17:58.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93354694600001, 2024-09-21T23:17:58.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:17:58.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93349080600001, 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:17:58.887 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:17:58.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93343702300001, 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:17:58.892 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:17:58.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.897 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.933404747, 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:17:58.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:17:58.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93337446699999, 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:17:58.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:17:58.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93334481100001, 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.912 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93331484399999, 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.913 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:17:58.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:17:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:17:58.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.917 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.933237015, 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:17:58.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:17:58.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93318930800001, 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.927 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.933159112, 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.928 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:17:58.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:17:58.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.932 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93312919499999, 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:58.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:17:58.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.937 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.933099395, 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:58.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:17:58.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93306946199999, 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.943 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:17:58.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93300449899999, 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:17:58.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:17:58.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:17:58.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.93297402300001, 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:17:58.957 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.932944859, 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:58.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:17:58.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:17:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:17:58.962 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:17:58.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:17:58.965 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:58.964+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:59.066 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:17:59.066+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:17:59.096 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:59.096 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:59.111 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:17:59.167 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:59.164+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:59.229 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:59.229 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:59.229 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:59.229 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:59.229 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:59.243 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:17:59.286 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:17:59.284+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:17:59.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:17:59.368 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:59.364+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:59.419 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:59.419 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 193.19634586500001, 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:17:59.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:17:59.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.426 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.68394058199999, 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:59.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:17:59.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:17:59.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.68387567900001, 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.437 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.68372923999999, 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:17:59.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:17:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.68366012499999, 2024-09-21T23:17:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:17:59.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:17:59.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:17:59.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:17:59.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.6830478, 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:17:59.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:17:59.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.68284339499999, 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:17:59.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:17:59.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:17:59.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:17:59.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:17:59.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.461 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.461 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.682800609, 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.462 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:17:59.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:17:59.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.467 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.29184232099999, 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:59.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:17:59.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.291755811, 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.473 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:17:59.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.29167892199999, 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:17:59.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:17:59.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.29147408200001, 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.485 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:17:59.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:17:59.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.2913858, 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:17:59.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:17:59.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.496 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.29125497199999, 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:17:59.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:17:59.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:17:59.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:17:59.501 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.29120931899999, 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.502 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.507 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.291170892, 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.29104143000001, 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:17:59.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:17:59.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.518 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.291019788, 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:17:59.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:59.524 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.290999744, 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:17:59.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:17:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 192.29091575199999, 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:17:59.530 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:17:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:17:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:17:59.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:17:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:17:59.535 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:59.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:17:59.537 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:17:59.532+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:17:59.637 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:59.634+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:17:59.710 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:59.710 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:59.710 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:17:59.710 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:59.724 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:17:59.773 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:59.772+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:59.840 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:17:59.836+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:17:59.847 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:17:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:17:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:17:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:17:59.847 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:17:59.861 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:17:59.940 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:17:59.942+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:00.028 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 145.07022548500001, 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:18:00.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:18:00.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:18:00.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 145.069958501, 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:18:00.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:18:00.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:00.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.45762995600001, 2024-09-21T23:18:00.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:00.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.457587257, 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:18:00.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:18:00.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.457548637, 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:18:00.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:18:00.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:00.057 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.45751676499999, 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:18:00.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:18:00.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.457483541, 2024-09-21T23:18:00.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:18:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:18:00.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.457448466, 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:18:00.067 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:18:00.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.072 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.072 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:00.072 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.457403508, 2024-09-21T23:18:00.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:18:00.073 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.073 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.073 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:18:00.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:18:00.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.456951441, 2024-09-21T23:18:00.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:00.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:18:00.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:18:00.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.45623696200001, 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:18:00.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.086 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.456195892, 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:18:00.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.455738774, 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:18:00.091 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:18:00.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:18:00.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.455502095, 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.096 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:18:00.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.455086514, 2024-09-21T23:18:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:18:00.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.104 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.45440885399999, 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.105 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:18:00.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.109 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.453917565, 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.113 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 103.45366066699999, 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:00.114 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:00.118 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 46.744469568, 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:18:00.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 46.741590866999999, 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.123 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:18:00.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:18:00.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:18:00.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:18:00.129 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.129 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.129 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.129 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.129 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.129 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:00.130 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:00.131 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:00.128+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:00.231 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:00.228+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:00.290 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:00.290 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:00.291 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:00.291 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:00.304 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:00.310 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:18:00.345 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:00.344+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:00.406 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:18:00.406 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:18:00.406 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T23:18:00.406 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:18:00.406 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:18:00.407 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-21T23:18:00.457 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:00.456+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:00.476 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:00.476 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:00.476 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:00.476 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:00.476 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:00.492 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:00.531 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:00.532 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "" 2024-09-21T23:18:00.532 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:00.533 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:00.532+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:00.634 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:00.632+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:00.652 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.399615013000002, 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:18:00.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:18:00.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:18:00.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.658 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.374566053999999, 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:18:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:18:00.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.341132883999997, 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:18:00.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:18:00.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1238 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.370382423999999, 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054982073999999999, 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1238 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:00.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280879+0000", 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8700000000000005e-07 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280883+0000", 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.912e-06 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280894+0000", 2024-09-21T23:18:00.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1275e-05 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280904+0000", 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.73e-06 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335407+0000", 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054502816000000003 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335499+0000", 2024-09-21T23:18:00.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2589000000000004e-05 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335860+0000", 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036096499999999998 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1341 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.366950916, 2024-09-21T23:18:00.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055656854999999998, 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1341 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284311+0000", 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.06e-06 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284314+0000", 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.72e-06 2024-09-21T23:18:00.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284328+0000", 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3380000000000001e-05 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284338+0000", 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0052e-05 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338191+0000", 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053852786999999999 2024-09-21T23:18:00.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338273+0000", 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2021000000000006e-05 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339967+0000", 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016938350000000001 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1342 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 966656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.677 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.366910936, 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056087662000000003, 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1342 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284354+0000", 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.569e-06 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284364+0000", 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0710000000000001e-05 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284373+0000", 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7819999999999999e-06 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340014+0000", 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055641057000000001 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340135+0000", 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120698 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340437+0000", 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030229200000000001 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.681 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1343 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.366872953000001, 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055970117, 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.682 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1343 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9200000000000001e-07 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284392+0000", 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8240000000000001e-06 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284404+0000", 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1584e-05 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284415+0000", 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1412e-05 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339993+0000", 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055577519999999998 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340056+0000", 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3127999999999996e-05 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340358+0000", 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000302057 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1344 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.366833588999999, 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056410920000000003, 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.686 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1344 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284428+0000", 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284432+0000", 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9060000000000004e-06 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284443+0000", 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.134e-05 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284452+0000", 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0340000000000002e-06 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340391+0000", 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055938437000000001 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340464+0000", 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3744000000000004e-05 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340838+0000", 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000373595 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1345 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.366797988999998, 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056714902999999997, 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1345 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:00.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284464+0000", 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284467+0000", 2024-09-21T23:18:00.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.63e-06 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284479+0000", 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.17e-05 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284488+0000", 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1870000000000003e-06 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340886+0000", 2024-09-21T23:18:00.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056398177000000001 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340951+0000", 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4263000000000005e-05 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341178+0000", 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022679199999999999 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.694 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1346 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.366731516000002, 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057046119999999999, 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.695 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1346 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284530+0000", 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284535+0000", 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7960000000000003e-06 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284548+0000", 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3002e-05 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284558+0000", 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8640000000000008e-06 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341212+0000", 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056654228000000001 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341281+0000", 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9061999999999997e-05 2024-09-21T23:18:00.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341575+0000", 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000294152 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1419 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 32768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.365883956000001, 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056364545000000002, 2024-09-21T23:18:00.699 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1419 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285381+0000", 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:18:00.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285391+0000", 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.03e-05 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285400+0000", 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1959999999999997e-06 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341615+0000", 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056215380000000002 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341674+0000", 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8403000000000003e-05 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341741+0000", 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7352000000000002e-05 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1420 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.365847943999999, 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056531351000000001, 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1420 2024-09-21T23:18:00.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:00.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3799999999999997e-07 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285417+0000", 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0459999999999999e-06 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285429+0000", 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1402e-05 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285438+0000", 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2050000000000008e-06 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341791+0000", 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056353542999999999 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341861+0000", 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9338000000000006e-05 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341944+0000", 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3178999999999998e-05 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1421 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.365808245, 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056621478000000003, 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.708 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1421 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:00.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285453+0000", 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.066e-06 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285457+0000", 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285468+0000", 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0974e-05 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285477+0000", 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0459999999999994e-06 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341982+0000", 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056504285000000001 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342029+0000", 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7528999999999998e-05 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342074+0000", 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4647999999999997e-05 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1424 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.363302121, 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054910207000000003, 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1424 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287959+0000", 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0139999999999999e-06 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287965+0000", 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287989+0000", 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4511e-05 2024-09-21T23:18:00.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288003+0000", 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3318e-05 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342685+0000", 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054682236000000002 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342764+0000", 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9122999999999994e-05 2024-09-21T23:18:00.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342869+0000", 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010467399999999999 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1448 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.362775920000001, 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055210594000000002, 2024-09-21T23:18:00.717 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1448 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288486+0000", 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4200000000000004e-07 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288490+0000", 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8979999999999998e-06 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288501+0000", 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288509+0000", 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7499999999999992e-06 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343373+0000", 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05486365 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343441+0000", 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8119000000000003e-05 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343695+0000", 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025414099999999999 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1449 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.721 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.362739888, 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055644921999999999, 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1449 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4600000000000005e-07 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288525+0000", 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9519999999999996e-06 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288536+0000", 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0862e-05 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288544+0000", 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.276e-06 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343752+0000", 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055208113000000003 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343837+0000", 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4466000000000001e-05 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344166+0000", 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032870700000000002 2024-09-21T23:18:00.725 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1450 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.362704736000001, 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055497638000000002, 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.726 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1450 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288557+0000", 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288562+0000", 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7860000000000002e-06 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288574+0000", 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1637999999999999e-05 2024-09-21T23:18:00.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288583+0000", 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8240000000000006e-06 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343733+0000", 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055150504000000003 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343785+0000", 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1805999999999998e-05 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344054+0000", 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026842000000000001 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1451 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.362666308000001, 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055892521000000001, 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1451 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288595+0000", 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288599+0000", 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1999999999999996e-06 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288611+0000", 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1506e-05 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288619+0000", 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7790000000000001e-06 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344089+0000", 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055469472999999998 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344170+0000", 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1287999999999998e-05 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344487+0000", 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031666499999999999 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1452 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.362630126999999, 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056290277999999999, 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.735 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1452 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288631+0000", 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288635+0000", 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0339999999999998e-06 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288646+0000", 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1107999999999999e-05 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288656+0000", 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3319999999999998e-06 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344531+0000", 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055875073999999997 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344606+0000", 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4838000000000004e-05 2024-09-21T23:18:00.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344921+0000", 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031523599999999999 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1453 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.362594270999999, 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056627027000000003, 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:00.739 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1453 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:00.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288667+0000", 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288671+0000", 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.293991+0000", 2024-09-21T23:18:00.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053204910000000001 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.294012+0000", 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1019999999999999e-05 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344962+0000", 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050949529 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345048+0000", 2024-09-21T23:18:00.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6263999999999999e-05 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345293+0000", 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000245135 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:00.743 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:00.745 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:00.745 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:00.746+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:00.846 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:00.844+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:00.881 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:00.881 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:00.897 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:00.947 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:00.946+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:00.963 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:18:01.028 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:01.028 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:01.028 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:01.028 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:01.028 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:01.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:01.087 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:01.086+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:01.102 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:01.119 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:01.119 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:01.136 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:01.136 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:01.149 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:01.146+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:01.157 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:01.157 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:01.166 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:01.166 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:01.178 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:01.179 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:01.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:01.200 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:01.201 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:01.267 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:01.266+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:01.325 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:01.325 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:01.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:01.387 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:01.386+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:01.452 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:01.448+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:01.521 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:01.521 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:01.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:01.553 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:01.548+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:01.654 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:01.650+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:01.668 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:01.668 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:01.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:01.758 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:01.752+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:01.856 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:01.852+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:01.871 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:01.871 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:01.887 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:01.957 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:01.954+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:02.017 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:02.017 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:02.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:02.058 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:02.056+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:02.159 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:02.156+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:02.202 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:02.202 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:02.218 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:02.262 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:02.260+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:02.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:18:02.350 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:02.350 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:02.365 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:02.391 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:02.394+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:02.462 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:02.460+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:02.522 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:02.523 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:02.537 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:02.582 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:02.580+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:02.663 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:02.660+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:02.699 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:02.700 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:02.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:18:02.764 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:02.762+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:02.865 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:02.864+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:02.888 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:02.888 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:02.903 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:02.966 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:02.966+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:03.066 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:03.066+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:03.073 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:03.073 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:03.087 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:03.167 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:03.170+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:03.211 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:03.211 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:03.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:18:03.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:03.268 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:03.270+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:03.348 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:03.348 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:03.362 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:03.368 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:03.370+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:03.381 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:03.381 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:03.397 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:03.398 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:03.414 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:03.414 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:03.432 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:03.432 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:03.452 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:03.452 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:03.470 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:03.470+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:03.472 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:03.487 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:03.487 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:03.488 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:03.501 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:03.571 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:03.568+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:03.626 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:03.626 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.371506826999997, 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:03.627 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:18:03.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:18:03.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.346457868000002, 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:03.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:03.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:18:03.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.313024698, 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1238 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.342274238000002, 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054982073999999999, 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1238 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280879+0000", 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8700000000000005e-07 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280883+0000", 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.912e-06 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280894+0000", 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1275e-05 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280904+0000", 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.73e-06 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335407+0000", 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054502816000000003 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335499+0000", 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2589000000000004e-05 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335860+0000", 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036096499999999998 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1341 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.33884273, 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055656854999999998, 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1341 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284311+0000", 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.06e-06 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284314+0000", 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.72e-06 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284328+0000", 2024-09-21T23:18:03.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3380000000000001e-05 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284338+0000", 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0052e-05 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338191+0000", 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053852786999999999 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338273+0000", 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2021000000000006e-05 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339967+0000", 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016938350000000001 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1342 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 966656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.338802749999999, 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056087662000000003, 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1342 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:03.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284354+0000", 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.569e-06 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284364+0000", 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0710000000000001e-05 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284373+0000", 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7819999999999999e-06 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340014+0000", 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055641057000000001 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340135+0000", 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120698 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340437+0000", 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030229200000000001 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1343 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.338764767000001, 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055970117, 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.656 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1343 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:03.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9200000000000001e-07 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284392+0000", 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8240000000000001e-06 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284404+0000", 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1584e-05 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284415+0000", 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1412e-05 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339993+0000", 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055577519999999998 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340056+0000", 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3127999999999996e-05 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340358+0000", 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000302057 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1344 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.338725403000002, 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056410920000000003, 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1344 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284428+0000", 2024-09-21T23:18:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284432+0000", 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9060000000000004e-06 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284443+0000", 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.134e-05 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284452+0000", 2024-09-21T23:18:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0340000000000002e-06 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340391+0000", 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055938437000000001 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340464+0000", 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3744000000000004e-05 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340838+0000", 2024-09-21T23:18:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000373595 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1345 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.338689803000001, 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056714902999999997, 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1345 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284464+0000", 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284467+0000", 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.63e-06 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284479+0000", 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.17e-05 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284488+0000", 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1870000000000003e-06 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340886+0000", 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056398177000000001 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340951+0000", 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4263000000000005e-05 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341178+0000", 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022679199999999999 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1346 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.338623330000001, 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057046119999999999, 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1346 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284530+0000", 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284535+0000", 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7960000000000003e-06 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284548+0000", 2024-09-21T23:18:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3002e-05 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284558+0000", 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8640000000000008e-06 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341212+0000", 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056654228000000001 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341281+0000", 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9061999999999997e-05 2024-09-21T23:18:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341575+0000", 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000294152 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1419 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 32768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.33777577, 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056364545000000002, 2024-09-21T23:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1419 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285381+0000", 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:18:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285391+0000", 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.03e-05 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285400+0000", 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1959999999999997e-06 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341615+0000", 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056215380000000002 2024-09-21T23:18:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341674+0000", 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8403000000000003e-05 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341741+0000", 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7352000000000002e-05 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1420 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.337739758000001, 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056531351000000001, 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1420 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3799999999999997e-07 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285417+0000", 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0459999999999999e-06 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285429+0000", 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1402e-05 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285438+0000", 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2050000000000008e-06 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341791+0000", 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056353542999999999 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341861+0000", 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9338000000000006e-05 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341944+0000", 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3178999999999998e-05 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1421 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.337700058999999, 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056621478000000003, 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1421 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285453+0000", 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.066e-06 2024-09-21T23:18:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285457+0000", 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285468+0000", 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0974e-05 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285477+0000", 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0459999999999994e-06 2024-09-21T23:18:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341982+0000", 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056504285000000001 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342029+0000", 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7528999999999998e-05 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342074+0000", 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4647999999999997e-05 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1424 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.335193934999999, 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054910207000000003, 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1424 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287959+0000", 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0139999999999999e-06 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287965+0000", 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287989+0000", 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4511e-05 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288003+0000", 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3318e-05 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342685+0000", 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054682236000000002 2024-09-21T23:18:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342764+0000", 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9122999999999994e-05 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342869+0000", 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010467399999999999 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1448 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.334667734, 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055210594000000002, 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1448 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288486+0000", 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4200000000000004e-07 2024-09-21T23:18:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288490+0000", 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8979999999999998e-06 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288501+0000", 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288509+0000", 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7499999999999992e-06 2024-09-21T23:18:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343373+0000", 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05486365 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343441+0000", 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8119000000000003e-05 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343695+0000", 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025414099999999999 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1449 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.334631701999999, 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055644921999999999, 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1449 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4600000000000005e-07 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288525+0000", 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9519999999999996e-06 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288536+0000", 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0862e-05 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288544+0000", 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.276e-06 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343752+0000", 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055208113000000003 2024-09-21T23:18:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343837+0000", 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4466000000000001e-05 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344166+0000", 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032870700000000002 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1450 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.334596550000001, 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055497638000000002, 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1450 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288557+0000", 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288562+0000", 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7860000000000002e-06 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288574+0000", 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1637999999999999e-05 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288583+0000", 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8240000000000006e-06 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343733+0000", 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055150504000000003 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343785+0000", 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1805999999999998e-05 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344054+0000", 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026842000000000001 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1451 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.334558122000001, 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055892521000000001, 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1451 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288595+0000", 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:18:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288599+0000", 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1999999999999996e-06 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288611+0000", 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1506e-05 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288619+0000", 2024-09-21T23:18:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7790000000000001e-06 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344089+0000", 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055469472999999998 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344170+0000", 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1287999999999998e-05 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344487+0000", 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031666499999999999 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1452 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.334521940999998, 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056290277999999999, 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1452 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288631+0000", 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288635+0000", 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0339999999999998e-06 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288646+0000", 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1107999999999999e-05 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288656+0000", 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3319999999999998e-06 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344531+0000", 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055875073999999997 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344606+0000", 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4838000000000004e-05 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344921+0000", 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031523599999999999 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1453 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.334486085000002, 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056627027000000003, 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1453 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288667+0000", 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288671+0000", 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.293991+0000", 2024-09-21T23:18:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053204910000000001 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.294012+0000", 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1019999999999999e-05 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344962+0000", 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050949529 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345048+0000", 2024-09-21T23:18:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6263999999999999e-05 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345293+0000", 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000245135 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:03.717 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:03.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:03.719 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:03.722+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:03.820 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:03.822+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:03.863 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:03.863 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:03.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:03.921 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:03.922+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:04.021 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:04.016+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:04.049 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:04.049 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:04.049 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:04.049 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:04.049 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:04.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:04.122 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:04.120+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:04.194 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:18:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.03264802, 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:18:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.967393262, 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:18:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:18:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:18:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 112.57005113, 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:18:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 112.566683441, 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:18:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:18:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:18:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 112.562573861, 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 112.55878094000001, 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494865062, 2024-09-21T23:18:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:18:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494838261, 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494812096, 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:04.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:18:04.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:18:04.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494783646, 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:18:04.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494692938, 2024-09-21T23:18:04.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.49466135900001, 2024-09-21T23:18:04.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.248 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.49462366900001, 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494593279, 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:04.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:18:04.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494549862, 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:18:04.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:18:04.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.494515237, 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:18:04.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.266 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.491261737, 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.491227155, 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.275482756, 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:04.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:04.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:18:04.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:18:04.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:18:04.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:18:04.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.666507559999999, 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:18:04.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.288 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.288 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:04.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:04.290 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:04.288+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:04.390 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:04.390+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:04.420 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:04.420 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:04.420 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:04.420 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:04.434 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:04.519 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:04.522+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:04.592 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:04.594+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:04.606 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:04.606 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:04.606 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:04.606 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:04.606 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:04.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:04.711 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:04.714+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:04.793 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:04.792+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:04.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:18:04.803 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:04.803 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:04.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:04.803 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:04.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.042043147, 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.04199789, 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041948268, 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:04.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041918573, 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:04.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:18:04.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:18:04.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:04.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.04188564499999, 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:04.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041851349, 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:18:04.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:18:04.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:18:04.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:18:04.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041826843, 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:18:04.831 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:04.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:18:04.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:18:04.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041798853, 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041775825, 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.841 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:18:04.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.04175431500001, 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041718774, 2024-09-21T23:18:04.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:18:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041691137, 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.854 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041655973, 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.858 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:04.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:18:04.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:18:04.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.04162094199999, 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.863 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:18:04.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041585557, 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:18:04.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:18:04.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041545375, 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.872 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:18:04.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041508247, 2024-09-21T23:18:04.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.04146817199999, 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:18:04.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:18:04.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:18:04.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.041443628, 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:18:04.885 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:18:04.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.889 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.04140584300001, 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:18:04.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:18:04.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:18:04.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:04.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:04.894 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:04.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:04.895 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:04.898+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:04.996 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:04.992+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:05.057 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:05.057 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:05.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:05.119 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:05.122+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:05.192 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:05.192 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:05.192 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:05.192 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:05.192 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:05.198 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:05.198+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:05.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:05.300 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:05.296+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.450891165, 2024-09-21T23:18:05.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:05.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:18:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44987031799999, 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:18:05.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44981064500001, 2024-09-21T23:18:05.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:18:05.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:18:05.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44972066899999, 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:18:05.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:18:05.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:18:05.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:18:05.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:18:05.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44967075900001, 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:18:05.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:18:05.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44961461899999, 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:18:05.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:18:05.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44956083599999, 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:18:05.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:18:05.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:18:05.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44952856, 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:18:05.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:18:05.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:18:05.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44949828, 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:18:05.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:18:05.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:18:05.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44946862399999, 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:18:05.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:18:05.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.449438657, 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:18:05.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44936082800001, 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:18:05.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:18:05.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44931312099999, 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:18:05.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44928292500001, 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.451 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:18:05.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.449253008, 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:18:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:18:05.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.461 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44922320800001, 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:18:05.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:18:05.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.449193275, 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.467 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:05.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:18:05.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.449128312, 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.472 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:18:05.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:18:05.476 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44909783599999, 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.477 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:18:05.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:18:05.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 150.44906867200001, 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:05.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:05.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:18:05.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:18:05.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:05.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:18:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:18:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:05.487 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:05.487 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:05.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:05.489 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:05.484+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:05.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:18:05.548 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:18:05.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:18:05.590 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:05.588+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:05.621 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:05.621 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:05.621 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:05.621 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:05.635 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:05.643 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:05.644 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:05.661 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:05.661 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:05.678 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:05.678 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:05.690 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:05.690+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:05.698 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:05.698 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:05.715 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:05.716 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:05.735 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:05.735 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:05.777 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:18:05.791 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:05.790+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:05.814 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:05.814 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:05.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:05.892 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:05.894+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:05.958 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:05.958 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:05.974 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:05.993 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:05.994+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:06.094 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:06.094+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:06.113 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:06.194 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:06.192+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:06.247 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:06.295 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:06.298+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:06.396 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:06.398+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:06.411 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:06.411 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:06.424 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:06.497 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:06.498+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:06.542 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:06.542 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:06.556 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:06.605 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:06.604+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:06.679 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:06.679 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:06.693 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:06.699 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:06.702+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:06.759 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:18:06.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:18:06.800 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:06.796+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:06.864 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:06.864 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:06.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:06.922 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:06.920+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:07.002 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:07.002+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:07.016 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:07.016 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:07.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:07.119 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:07.122+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:07.197 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:07.198 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:07.204 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:07.200+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:07.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:07.305 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:07.300+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:07.379 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:07.379 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:07.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:07.405 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:07.404+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:07.506 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:07.504+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:07.523 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:07.523 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:07.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:07.608 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:07.610+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:07.664 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:07.665 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:07.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:07.723 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:07.726+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:07.754 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:18:07.782 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:18:07.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:18:07.809 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:18:07.810 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:07.810+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:07.861 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:07.861 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:07.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:07.927 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:07.930+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:07.931 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:07.931 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:07.957 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:07.957 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:07.978 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:07.978 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:08.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:08.000 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:08.010 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:08.010+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:08.023 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:08.024 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:08.058 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:08.059 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:08.063 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:08.063 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:08.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:08.123 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:08.126+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:08.213 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:08.208+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:08.232 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:08.232 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:08.232 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:08.232 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:08.232 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:08.244 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:08.313 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:08.314+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:08.370 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.370 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.442386438, 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:18:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44136559099999, 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:18:08.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:18:08.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:18:08.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44130591800001, 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:18:08.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:18:08.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44121594200001, 2024-09-21T23:18:08.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:18:08.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44116603200001, 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:18:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44110989199999, 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.397 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44105610899999, 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:18:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.441023833, 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.407 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:18:08.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.440993553, 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:18:08.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:18:08.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44096389699999, 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:18:08.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:18:08.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:18:08.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44093393, 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:18:08.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:18:08.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44085610100001, 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.429 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:18:08.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44080839399999, 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:18:08.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:18:08.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:08.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.440778198, 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:18:08.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.440748281, 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:18:08.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:18:08.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.440718481, 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.450 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:18:08.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:18:08.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.455 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.440688548, 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.456 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:08.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:18:08.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:08.461 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.440623585, 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:08.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:18:08.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44059310899999, 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.467 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:18:08.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 153.44056394500001, 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:08.473 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:18:08.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:18:08.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:08.478 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:08.480 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:08.480 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:08.476+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:08.581 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:08.576+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:08.612 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:08.612 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:08.612 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:08.612 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:08.627 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:08.713 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:08.712+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:08.782 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:08.782+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:08.790 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:08.790 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:18:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:18:08.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:08.830 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:18:08.915 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:08.918+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:08.984 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:08.980+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.764266779, 2024-09-21T23:18:08.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:08.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:18:08.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:18:08.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:08.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:18:08.993 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.251861496, 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:08.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:08.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:18:08.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:08.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:08.999 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.25179659299999, 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:18:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:18:09.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.251650154, 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:18:09.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:18:09.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:18:09.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.011 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.251581039, 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.012 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:18:09.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:18:09.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.25096871400001, 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:18:09.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:18:09.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.250764309, 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:18:09.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:18:09.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.029 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 202.25072152300001, 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:18:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.035 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.859763235, 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:18:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:18:09.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.85967672500001, 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:18:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:18:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.859599836, 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:18:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.85939499599999, 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:18:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:18:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.85930671400001, 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:09.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:18:09.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:18:09.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.859175886, 2024-09-21T23:18:09.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:18:09.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.859130233, 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.071 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:09.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:18:09.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.076 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.85909180600001, 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.077 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:18:09.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:18:09.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.85896234399999, 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:18:09.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:18:09.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.088 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.85894070200001, 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:18:09.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:18:09.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.85892065799999, 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.095 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:09.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:18:09.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.100 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 201.858836666, 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.106 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:09.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:09.108 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:09.104+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:09.208 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:09.210+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:09.279 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:09.280 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:09.293 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:09.337 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:09.336+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:09.410 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:09.410+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:09.411 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:09.411 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:09.425 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:09.513 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:09.512+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:09.594 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 154.637259376, 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.595 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:18:09.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:09.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.600 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 154.636992392, 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:18:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.024663847, 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:18:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.02462114799999, 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.611 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:18:09.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.024582528, 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.616 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.024550656, 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.620 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:18:09.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.024517432, 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.625 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:18:09.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.024482357, 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.629 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.02443739900001, 2024-09-21T23:18:09.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:18:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.637 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.023985332, 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:18:09.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:09.642 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.023270853, 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:18:09.646 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.023229783, 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.651 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.02277266500001, 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:18:09.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:18:09.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:09.656 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.02253598599999, 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:09.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:18:09.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.022120405, 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:18:09.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:09.665 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.021442745, 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:18:09.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.02095145600001, 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:18:09.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:09.675 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.020694558, 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:18:09.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.311503459000001, 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.680 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:18:09.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:09.684 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.308624758000001, 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:18:09.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:18:09.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.690 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:09.692 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:09.692 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:09.688+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:09.793 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:09.788+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:09.799 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:18:09.830 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:18:09.830 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:26885 3.13 3:cab0a1e4:::rbd_object_map.10f1d740b9a0:head [call lock.unlock in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e159)", 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:18:09.868959+0000", 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00080097600000000001, 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087161400000000005, 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:09.871 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for sub ops", 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26885 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.868959+0000", 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:09.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.868959+0000", 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.868961+0000", 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.863e-06 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.868971+0000", 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0438999999999999e-05 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.869000+0000", 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9240000000000001e-05 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.869015+0000", 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.472e-05 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.869132+0000", 2024-09-21T23:18:09.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000116765 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.869277+0000", 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014571000000000001 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:09.869555+0000", 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00027705900000000002 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:09.876 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:18:09.876 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:18:09.876 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:09.886 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:09.930 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:09.928+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:09.995 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:09.994+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:10.059 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:10.059 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:10.074 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:10.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:18:10.123 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:10.126+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:10.196 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:10.198+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:10.202 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.953314421000002, 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:18:10.208 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:18:10.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.928265461999999, 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.214 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.219 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.894832291999997, 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:18:10.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:10.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1238 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:10.224 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.924081831999999, 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054982073999999999, 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1238 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280879+0000", 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8700000000000005e-07 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280883+0000", 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.912e-06 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280894+0000", 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1275e-05 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280904+0000", 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.73e-06 2024-09-21T23:18:10.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335407+0000", 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054502816000000003 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335499+0000", 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2589000000000004e-05 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335860+0000", 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036096499999999998 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1341 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.920650324, 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055656854999999998, 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.229 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1341 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284311+0000", 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.06e-06 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284314+0000", 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.72e-06 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284328+0000", 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3380000000000001e-05 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284338+0000", 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0052e-05 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338191+0000", 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053852786999999999 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338273+0000", 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2021000000000006e-05 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339967+0000", 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016938350000000001 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1342 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 966656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:10.233 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.920610344, 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056087662000000003, 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1342 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:10.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284354+0000", 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.569e-06 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284364+0000", 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0710000000000001e-05 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284373+0000", 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7819999999999999e-06 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340014+0000", 2024-09-21T23:18:10.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055641057000000001 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340135+0000", 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120698 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340437+0000", 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030229200000000001 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1343 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.920572361000001, 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055970117, 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1343 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9200000000000001e-07 2024-09-21T23:18:10.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284392+0000", 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8240000000000001e-06 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284404+0000", 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1584e-05 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284415+0000", 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1412e-05 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339993+0000", 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055577519999999998 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340056+0000", 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3127999999999996e-05 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340358+0000", 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000302057 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1344 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.920532996999999, 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056410920000000003, 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.242 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1344 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284428+0000", 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284432+0000", 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9060000000000004e-06 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284443+0000", 2024-09-21T23:18:10.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.134e-05 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284452+0000", 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0340000000000002e-06 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340391+0000", 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055938437000000001 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340464+0000", 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3744000000000004e-05 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340838+0000", 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000373595 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1345 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:10.246 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.920497396999998, 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056714902999999997, 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1345 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284464+0000", 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284467+0000", 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.63e-06 2024-09-21T23:18:10.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284479+0000", 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.17e-05 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284488+0000", 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1870000000000003e-06 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340886+0000", 2024-09-21T23:18:10.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056398177000000001 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340951+0000", 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4263000000000005e-05 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341178+0000", 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022679199999999999 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1346 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.920430924000001, 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057046119999999999, 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1346 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.251 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284530+0000", 2024-09-21T23:18:10.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284535+0000", 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7960000000000003e-06 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284548+0000", 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3002e-05 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284558+0000", 2024-09-21T23:18:10.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8640000000000008e-06 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341212+0000", 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056654228000000001 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341281+0000", 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9061999999999997e-05 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341575+0000", 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000294152 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1419 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 32768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.919583364000001, 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056364545000000002, 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1419 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285381+0000", 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285391+0000", 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.03e-05 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285400+0000", 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1959999999999997e-06 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341615+0000", 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056215380000000002 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341674+0000", 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8403000000000003e-05 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341741+0000", 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7352000000000002e-05 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1420 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.259 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.919547351999999, 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056531351000000001, 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1420 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3799999999999997e-07 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285417+0000", 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0459999999999999e-06 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285429+0000", 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1402e-05 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285438+0000", 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2050000000000008e-06 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341791+0000", 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056353542999999999 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341861+0000", 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9338000000000006e-05 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341944+0000", 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3178999999999998e-05 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.263 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1421 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.919507653, 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056621478000000003, 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.264 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1421 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285453+0000", 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.066e-06 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285457+0000", 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285468+0000", 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0974e-05 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285477+0000", 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0459999999999994e-06 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341982+0000", 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056504285000000001 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342029+0000", 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7528999999999998e-05 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342074+0000", 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4647999999999997e-05 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1424 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.917001529, 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054910207000000003, 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.268 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1424 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:10.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287959+0000", 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0139999999999999e-06 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287965+0000", 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287989+0000", 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4511e-05 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288003+0000", 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3318e-05 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342685+0000", 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054682236000000002 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342764+0000", 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9122999999999994e-05 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342869+0000", 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010467399999999999 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1448 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:10.272 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.916475328000001, 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055210594000000002, 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1448 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:10.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288486+0000", 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4200000000000004e-07 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288490+0000", 2024-09-21T23:18:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8979999999999998e-06 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288501+0000", 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288509+0000", 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7499999999999992e-06 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343373+0000", 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05486365 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343441+0000", 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8119000000000003e-05 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343695+0000", 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025414099999999999 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.276 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1449 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.916439296, 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055644921999999999, 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1449 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.277 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4600000000000005e-07 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288525+0000", 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9519999999999996e-06 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288536+0000", 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0862e-05 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288544+0000", 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.276e-06 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343752+0000", 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055208113000000003 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343837+0000", 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4466000000000001e-05 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344166+0000", 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032870700000000002 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1450 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.916404144000001, 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055497638000000002, 2024-09-21T23:18:10.281 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1450 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288557+0000", 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288562+0000", 2024-09-21T23:18:10.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7860000000000002e-06 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288574+0000", 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1637999999999999e-05 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288583+0000", 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8240000000000006e-06 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343733+0000", 2024-09-21T23:18:10.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055150504000000003 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343785+0000", 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1805999999999998e-05 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344054+0000", 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026842000000000001 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.285 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1451 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.916365716000001, 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055892521000000001, 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1451 2024-09-21T23:18:10.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288595+0000", 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288599+0000", 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1999999999999996e-06 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288611+0000", 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1506e-05 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288619+0000", 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7790000000000001e-06 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344089+0000", 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055469472999999998 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344170+0000", 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1287999999999998e-05 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344487+0000", 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031666499999999999 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1452 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.916329534999999, 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056290277999999999, 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.290 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1452 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:10.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288631+0000", 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288635+0000", 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0339999999999998e-06 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288646+0000", 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1107999999999999e-05 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288656+0000", 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3319999999999998e-06 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344531+0000", 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055875073999999997 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344606+0000", 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4838000000000004e-05 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344921+0000", 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031523599999999999 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1453 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:10.294 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.916293678999999, 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056627027000000003, 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1453 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:10.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288667+0000", 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288671+0000", 2024-09-21T23:18:10.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.293991+0000", 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053204910000000001 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.294012+0000", 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1019999999999999e-05 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344962+0000", 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050949529 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345048+0000", 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6263999999999999e-05 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345293+0000", 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000245135 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.298 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.299 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:10.299 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:10.299 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:10.300 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-21T23:18:09.801201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602906921,"num_objects":1285,"num_object_clones":323,"num_object_copies":2570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1285,"num_whiteouts":615,"num_read":306675,"num_read_kb":1073214,"num_write":20179,"num_write_kb":210299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2134,"num_bytes_recovered":24352285,"num_keys_recovered":317,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18421,"ondisk_log_size":18421,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":294748,"kb_used_data":74420,"kb_used_omap":10,"kb_used_meta":220213,"kb_avail":377192612,"statfs":{"total":386547056640,"available":386245234688,"internally_reserved":0,"allocated":76206080,"data_stored":81707683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":225498740},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47702190,"num_objects":-57,"num_object_clones":-25,"num_object_copies":-114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-57,"num_whiteouts":-18,"num_read":152,"num_read_kb":126,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025478"},"pg_stats":[{"pgid":"3.1f","version":"152'518","reported_seq":13187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.805444+0000","last_change":"2024-09-21T23:18:05.805444+0000","last_active":"2024-09-21T23:18:05.805444+0000","last_peered":"2024-09-21T23:18:05.805444+0000","last_clean":"2024-09-21T23:18:05.805444+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:18:05.805444+0000","last_undegraded":"2024-09-21T23:18:05.805444+0000","last_fullsized":"2024-09-21T23:18:05.805444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:19:10.321436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073146249999999999,"stat_sum":{"num_bytes":27626073,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":13294,"num_read_kb":21607,"num_write":587,"num_write_kb":3598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.1e","version":"152'447","reported_seq":5449,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838461+0000","last_change":"2024-09-21T23:18:04.838712+0000","last_active":"2024-09-21T23:18:04.838461+0000","last_peered":"2024-09-21T23:18:04.838461+0000","last_clean":"2024-09-21T23:18:04.838461+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:18:04.838461+0000","last_undegraded":"2024-09-21T23:18:04.838461+0000","last_fullsized":"2024-09-21T23:18:04.838461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:30:26.581188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093906199999999995,"stat_sum":{"num_bytes":22345283,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":5561,"num_read_kb":31938,"num_write":406,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1d","version":"153'569","reported_seq":7847,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.773961+0000","last_change":"2024-09-21T23:18:04.774395+0000","last_active":"2024-09-21T23:18:04.773961+0000","last_peered":"2024-09-21T23:18:04.773961+0000","last_clean":"2024-09-21T23:18:04.773961+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:04.773961+0000","last_undegraded":"2024-09-21T23:18:04.773961+0000","last_fullsized":"2024-09-21T23:18:04.773961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:14.088875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015720549999999999,"stat_sum":{"num_bytes":15382528,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":18,"num_read":8718,"num_read_kb":41550,"num_write":589,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1c","version":"154'901","reported_seq":12405,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.889490+0000","last_change":"2024-09-21T23:18:04.777771+0000","last_active":"2024-09-21T23:18:04.889490+0000","last_peered":"2024-09-21T23:18:04.889490+0000","last_clean":"2024-09-21T23:18:04.889490+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:04.889490+0000","last_undegraded":"2024-09-21T23:18:04.889490+0000","last_fullsized":"2024-09-21T23:18:04.889490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:51:38.857267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047225660000000001,"stat_sum":{"num_bytes":25558608,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":17022,"num_read_kb":70508,"num_write":948,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1b","version":"153'435","reported_seq":4981,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.795554+0000","last_change":"2024-09-21T23:18:05.795932+0000","last_active":"2024-09-21T23:18:05.795554+0000","last_peered":"2024-09-21T23:18:05.795554+0000","last_clean":"2024-09-21T23:18:05.795554+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:05.795554+0000","last_undegraded":"2024-09-21T23:18:05.795554+0000","last_fullsized":"2024-09-21T23:18:05.795554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:53:53.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670577,"stat_sum":{"num_bytes":11004483,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":5703,"num_read_kb":22515,"num_write":422,"num_write_kb":1615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.1a","version":"153'641","reported_seq":9952,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.817694+0000","last_change":"2024-09-21T23:18:04.783266+0000","last_active":"2024-09-21T23:18:04.817694+0000","last_peered":"2024-09-21T23:18:04.817694+0000","last_clean":"2024-09-21T23:18:04.817694+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:04.817694+0000","last_undegraded":"2024-09-21T23:18:04.817694+0000","last_fullsized":"2024-09-21T23:18:04.817694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:33.516022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320321,"stat_sum":{"num_bytes":8963107,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":11645,"num_read_kb":26350,"num_write":663,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.19","version":"152'380","reported_seq":5192,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837483+0000","last_change":"2024-09-21T23:18:04.837992+0000","last_active":"2024-09-21T23:18:04.837483+0000","last_peered":"2024-09-21T23:18:04.837483+0000","last_clean":"2024-09-21T23:18:04.837483+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:04.837483+0000","last_undegraded":"2024-09-21T23:18:04.837483+0000","last_fullsized":"2024-09-21T23:18:04.837483+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:37:46.052666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074324199999999997,"stat_sum":{"num_bytes":14995504,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":4913,"num_read_kb":66320,"num_write":312,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.18","version":"155'595","reported_seq":18302,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:06.020268+0000","last_change":"2024-09-21T23:18:05.801264+0000","last_active":"2024-09-21T23:18:06.020268+0000","last_peered":"2024-09-21T23:18:06.020268+0000","last_clean":"2024-09-21T23:18:06.020268+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:18:06.020268+0000","last_undegraded":"2024-09-21T23:18:06.020268+0000","last_fullsized":"2024-09-21T23:18:06.020268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:22:42.271596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052176990000000001,"stat_sum":{"num_bytes":43998796,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":16650,"num_read_kb":62664,"num_write":730,"num_write_kb":13737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.17","version":"155'382","reported_seq":13429,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.807148+0000","last_change":"2024-09-21T23:18:05.807148+0000","last_active":"2024-09-21T23:18:05.807148+0000","last_peered":"2024-09-21T23:18:05.807148+0000","last_clean":"2024-09-21T23:18:05.807148+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:05.807148+0000","last_undegraded":"2024-09-21T23:18:05.807148+0000","last_fullsized":"2024-09-21T23:18:05.807148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:31.915635+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0093331360000000006,"stat_sum":{"num_bytes":19626048,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":13363,"num_read_kb":35456,"num_write":325,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.16","version":"153'508","reported_seq":5684,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.775263+0000","last_change":"2024-09-21T23:18:04.775964+0000","last_active":"2024-09-21T23:18:04.775263+0000","last_peered":"2024-09-21T23:18:04.775263+0000","last_clean":"2024-09-21T23:18:04.775263+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:18:04.775263+0000","last_undegraded":"2024-09-21T23:18:04.775263+0000","last_fullsized":"2024-09-21T23:18:04.775263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:55.776886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028108439999999998,"stat_sum":{"num_bytes":44221602,"num_objects":76,"num_object_clones":27,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":41,"num_read":5526,"num_read_kb":23457,"num_write":567,"num_write_kb":3547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.15","version":"154'335","reported_seq":4069,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.804223+0000","last_change":"2024-09-21T23:18:05.804991+0000","last_active":"2024-09-21T23:18:05.804223+0000","last_peered":"2024-09-21T23:18:05.804223+0000","last_clean":"2024-09-21T23:18:05.804223+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:18:05.804223+0000","last_undegraded":"2024-09-21T23:18:05.804223+0000","last_fullsized":"2024-09-21T23:18:05.804223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:26:22.014694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071129269999999998,"stat_sum":{"num_bytes":20287488,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":3872,"num_read_kb":30392,"num_write":382,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.14","version":"151'853","reported_seq":7922,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.784458+0000","last_change":"2024-09-21T23:18:04.785083+0000","last_active":"2024-09-21T23:18:04.784458+0000","last_peered":"2024-09-21T23:18:04.784458+0000","last_clean":"2024-09-21T23:18:04.784458+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:18:04.784458+0000","last_undegraded":"2024-09-21T23:18:04.784458+0000","last_fullsized":"2024-09-21T23:18:04.784458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:44.895291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020148029999999999,"stat_sum":{"num_bytes":14212255,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":11159,"num_read_kb":36984,"num_write":1027,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":273,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.783375+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:18:04.783375+0000","last_peered":"2024-09-21T23:18:04.783375+0000","last_clean":"2024-09-21T23:18:04.783375+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:18:04.783375+0000","last_undegraded":"2024-09-21T23:18:04.783375+0000","last_fullsized":"2024-09-21T23:18:04.783375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"155'702","reported_seq":20008,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.804140+0000","last_change":"2024-09-21T23:18:05.804140+0000","last_active":"2024-09-21T23:18:05.804140+0000","last_peered":"2024-09-21T23:18:05.804140+0000","last_clean":"2024-09-21T23:18:05.804140+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:05.804140+0000","last_undegraded":"2024-09-21T23:18:05.804140+0000","last_fullsized":"2024-09-21T23:18:05.804140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:47:16.366244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0064937900000000002,"stat_sum":{"num_bytes":27026515,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":20805,"num_read_kb":33863,"num_write":997,"num_write_kb":4689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":176,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837862+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:18:04.837862+0000","last_peered":"2024-09-21T23:18:04.837862+0000","last_clean":"2024-09-21T23:18:04.837862+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:18:04.837862+0000","last_undegraded":"2024-09-21T23:18:04.837862+0000","last_fullsized":"2024-09-21T23:18:04.837862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"153'1614","reported_seq":8335,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838317+0000","last_change":"2024-09-21T23:18:04.838581+0000","last_active":"2024-09-21T23:18:04.838317+0000","last_peered":"2024-09-21T23:18:04.838317+0000","last_clean":"2024-09-21T23:18:04.838317+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:18:04.838317+0000","last_undegraded":"2024-09-21T23:18:04.838317+0000","last_fullsized":"2024-09-21T23:18:04.838317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1614,"log_dups_size":0,"ondisk_log_size":1614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:29.861515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089964299999999999,"stat_sum":{"num_bytes":11518890,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":13322,"num_read_kb":49388,"num_write":2770,"num_write_kb":8366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.797736+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:18:05.797736+0000","last_peered":"2024-09-21T23:18:05.797736+0000","last_clean":"2024-09-21T23:18:05.797736+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:18:05.797736+0000","last_undegraded":"2024-09-21T23:18:05.797736+0000","last_fullsized":"2024-09-21T23:18:05.797736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"151'404","reported_seq":2561,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837422+0000","last_change":"2024-09-21T23:18:04.837774+0000","last_active":"2024-09-21T23:18:04.837422+0000","last_peered":"2024-09-21T23:18:04.837422+0000","last_clean":"2024-09-21T23:18:04.837422+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:18:04.837422+0000","last_undegraded":"2024-09-21T23:18:04.837422+0000","last_fullsized":"2024-09-21T23:18:04.837422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:59:29.679115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055546199999999995,"stat_sum":{"num_bytes":8433438,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":2773,"num_read_kb":10789,"num_write":433,"num_write_kb":1935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837075+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:04.837075+0000","last_peered":"2024-09-21T23:18:04.837075+0000","last_clean":"2024-09-21T23:18:04.837075+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:04.837075+0000","last_undegraded":"2024-09-21T23:18:04.837075+0000","last_fullsized":"2024-09-21T23:18:04.837075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"153'478","reported_seq":10360,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.778031+0000","last_change":"2024-09-21T23:18:04.778760+0000","last_active":"2024-09-21T23:18:04.778031+0000","last_peered":"2024-09-21T23:18:04.778031+0000","last_clean":"2024-09-21T23:18:04.778031+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:18:04.778031+0000","last_undegraded":"2024-09-21T23:18:04.778031+0000","last_fullsized":"2024-09-21T23:18:04.778031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:04.376759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050390970000000002,"stat_sum":{"num_bytes":37486789,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":11101,"num_read_kb":76330,"num_write":579,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.3","version":"154'630","reported_seq":6550,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.904088+0000","last_change":"2024-09-21T23:18:04.784661+0000","last_active":"2024-09-21T23:18:04.904088+0000","last_peered":"2024-09-21T23:18:04.904088+0000","last_clean":"2024-09-21T23:18:04.904088+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:04.904088+0000","last_undegraded":"2024-09-21T23:18:04.904088+0000","last_fullsized":"2024-09-21T23:18:04.904088+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:05:38.872903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001617786,"stat_sum":{"num_bytes":13123137,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":8001,"num_read_kb":28761,"num_write":625,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.2","version":"19'2","reported_seq":312,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.772295+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:04.772295+0000","last_peered":"2024-09-21T23:18:04.772295+0000","last_clean":"2024-09-21T23:18:04.772295+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:04.772295+0000","last_undegraded":"2024-09-21T23:18:04.772295+0000","last_fullsized":"2024-09-21T23:18:04.772295+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"153'320","reported_seq":6089,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838662+0000","last_change":"2024-09-21T23:18:04.838908+0000","last_active":"2024-09-21T23:18:04.838662+0000","last_peered":"2024-09-21T23:18:04.838662+0000","last_clean":"2024-09-21T23:18:04.838662+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:18:04.838662+0000","last_undegraded":"2024-09-21T23:18:04.838662+0000","last_fullsized":"2024-09-21T23:18:04.838662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:23.240446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061421999999999996,"stat_sum":{"num_bytes":14368372,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":5593,"num_read_kb":23746,"num_write":279,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":277,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837085+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:18:04.837085+0000","last_peered":"2024-09-21T23:18:04.837085+0000","last_clean":"2024-09-21T23:18:04.837085+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:18:04.837085+0000","last_undegraded":"2024-09-21T23:18:04.837085+0000","last_fullsized":"2024-09-21T23:18:04.837085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"155'379","reported_seq":4535,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.803347+0000","last_change":"2024-09-21T23:18:05.803347+0000","last_active":"2024-09-21T23:18:05.803347+0000","last_peered":"2024-09-21T23:18:05.803347+0000","last_clean":"2024-09-21T23:18:05.803347+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:05.803347+0000","last_undegraded":"2024-09-21T23:18:05.803347+0000","last_fullsized":"2024-09-21T23:18:05.803347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:54.241364+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079002099999999995,"stat_sum":{"num_bytes":12197964,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":3901,"num_read_kb":15449,"num_write":357,"num_write_kb":5914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837758+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:18:04.837758+0000","last_peered":"2024-09-21T23:18:04.837758+0000","last_clean":"2024-09-21T23:18:04.837758+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:18:04.837758+0000","last_undegraded":"2024-09-21T23:18:04.837758+0000","last_fullsized":"2024-09-21T23:18:04.837758+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"153'589","reported_seq":16660,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837956+0000","last_change":"2024-09-21T23:18:04.838211+0000","last_active":"2024-09-21T23:18:04.837956+0000","last_peered":"2024-09-21T23:18:04.837956+0000","last_clean":"2024-09-21T23:18:04.837956+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:04.837956+0000","last_undegraded":"2024-09-21T23:18:04.837956+0000","last_fullsized":"2024-09-21T23:18:04.837956+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:47.575390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071451299999999995,"stat_sum":{"num_bytes":20596400,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":26,"num_read":18573,"num_read_kb":53288,"num_write":489,"num_write_kb":4973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.3","version":"17'1","reported_seq":299,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.773238+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:18:04.773238+0000","last_peered":"2024-09-21T23:18:04.773238+0000","last_clean":"2024-09-21T23:18:04.773238+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:18:04.773238+0000","last_undegraded":"2024-09-21T23:18:04.773238+0000","last_fullsized":"2024-09-21T23:18:04.773238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":391,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.795824+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:18:05.795824+0000","last_peered":"2024-09-21T23:18:05.795824+0000","last_clean":"2024-09-21T23:18:05.795824+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:18:05.795824+0000","last_undegraded":"2024-09-21T23:18:05.795824+0000","last_fullsized":"2024-09-21T23:18:05.795824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"153'857","reported_seq":12581,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838128+0000","last_change":"2024-09-21T23:18:04.838398+0000","last_active":"2024-09-21T23:18:04.838128+0000","last_peered":"2024-09-21T23:18:04.838128+0000","last_clean":"2024-09-21T23:18:04.838128+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:18:04.838128+0000","last_undegraded":"2024-09-21T23:18:04.838128+0000","last_fullsized":"2024-09-21T23:18:04.838128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.353282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084337699999999997,"stat_sum":{"num_bytes":30689824,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":13410,"num_read_kb":34760,"num_write":746,"num_write_kb":4277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.9","version":"153'487","reported_seq":4555,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.783670+0000","last_change":"2024-09-21T23:18:04.784322+0000","last_active":"2024-09-21T23:18:04.783670+0000","last_peered":"2024-09-21T23:18:04.783670+0000","last_clean":"2024-09-21T23:18:04.783670+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:18:04.783670+0000","last_undegraded":"2024-09-21T23:18:04.783670+0000","last_fullsized":"2024-09-21T23:18:04.783670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:38:54.406022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019839620000000001,"stat_sum":{"num_bytes":14514903,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":5596,"num_read_kb":14366,"num_write":396,"num_write_kb":2871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.a","version":"153'309","reported_seq":4011,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.783513+0000","last_change":"2024-09-21T23:18:04.783975+0000","last_active":"2024-09-21T23:18:04.783513+0000","last_peered":"2024-09-21T23:18:04.783513+0000","last_clean":"2024-09-21T23:18:04.783513+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:18:04.783513+0000","last_undegraded":"2024-09-21T23:18:04.783513+0000","last_fullsized":"2024-09-21T23:18:04.783513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:40:37.823983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020053599999999999,"stat_sum":{"num_bytes":13107216,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":3301,"num_read_kb":18163,"num_write":271,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.b","version":"153'695","reported_seq":8055,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.800218+0000","last_change":"2024-09-21T23:18:05.800827+0000","last_active":"2024-09-21T23:18:05.800218+0000","last_peered":"2024-09-21T23:18:05.800218+0000","last_clean":"2024-09-21T23:18:05.800218+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:05.800218+0000","last_undegraded":"2024-09-21T23:18:05.800218+0000","last_fullsized":"2024-09-21T23:18:05.800218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:31:04.326971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051601010000000003,"stat_sum":{"num_bytes":13150751,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":10071,"num_read_kb":21860,"num_write":636,"num_write_kb":2105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.c","version":"153'602","reported_seq":5693,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.777002+0000","last_change":"2024-09-21T23:18:04.777781+0000","last_active":"2024-09-21T23:18:04.777002+0000","last_peered":"2024-09-21T23:18:04.777002+0000","last_clean":"2024-09-21T23:18:04.777002+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:18:04.777002+0000","last_undegraded":"2024-09-21T23:18:04.777002+0000","last_fullsized":"2024-09-21T23:18:04.777002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:53:40.444767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043752310000000003,"stat_sum":{"num_bytes":18473517,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":7505,"num_read_kb":17833,"num_write":458,"num_write_kb":3340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.d","version":"147'440","reported_seq":5150,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.785006+0000","last_change":"2024-09-21T23:18:04.785717+0000","last_active":"2024-09-21T23:18:04.785006+0000","last_peered":"2024-09-21T23:18:04.785006+0000","last_clean":"2024-09-21T23:18:04.785006+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:18:04.785006+0000","last_undegraded":"2024-09-21T23:18:04.785006+0000","last_fullsized":"2024-09-21T23:18:04.785006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:25:06.653816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002095619,"stat_sum":{"num_bytes":7152396,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":6155,"num_read_kb":29381,"num_write":513,"num_write_kb":1707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.e","version":"154'410","reported_seq":4238,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.776859+0000","last_change":"2024-09-21T23:18:04.776859+0000","last_active":"2024-09-21T23:18:04.776859+0000","last_peered":"2024-09-21T23:18:04.776859+0000","last_clean":"2024-09-21T23:18:04.776859+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:04.776859+0000","last_undegraded":"2024-09-21T23:18:04.776859+0000","last_fullsized":"2024-09-21T23:18:04.776859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:54:29.165449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035514320000000002,"stat_sum":{"num_bytes":9538126,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":5265,"num_read_kb":14673,"num_write":376,"num_write_kb":2506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.f","version":"153'410","reported_seq":6205,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.778180+0000","last_change":"2024-09-21T23:18:04.778752+0000","last_active":"2024-09-21T23:18:04.778180+0000","last_peered":"2024-09-21T23:18:04.778180+0000","last_clean":"2024-09-21T23:18:04.778180+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:18:04.778180+0000","last_undegraded":"2024-09-21T23:18:04.778180+0000","last_fullsized":"2024-09-21T23:18:04.778180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:05.577891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004735052,"stat_sum":{"num_bytes":13897780,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":7493,"num_read_kb":34426,"num_write":376,"num_write_kb":4961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.10","version":"153'747","reported_seq":5949,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.773356+0000","last_change":"2024-09-21T23:18:04.774281+0000","last_active":"2024-09-21T23:18:04.773356+0000","last_peered":"2024-09-21T23:18:04.773356+0000","last_clean":"2024-09-21T23:18:04.773356+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:04.773356+0000","last_undegraded":"2024-09-21T23:18:04.773356+0000","last_fullsized":"2024-09-21T23:18:04.773356+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:01:05.646725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001506082,"stat_sum":{"num_bytes":14405691,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":7081,"num_read_kb":28619,"num_write":914,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.11","version":"153'498","reported_seq":5335,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.782342+0000","last_change":"2024-09-21T23:18:04.783135+0000","last_active":"2024-09-21T23:18:04.782342+0000","last_peered":"2024-09-21T23:18:04.782342+0000","last_clean":"2024-09-21T23:18:04.782342+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:18:04.782342+0000","last_undegraded":"2024-09-21T23:18:04.782342+0000","last_fullsized":"2024-09-21T23:18:04.782342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:29.093747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262399,"stat_sum":{"num_bytes":16721445,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":5625,"num_read_kb":28821,"num_write":632,"num_write_kb":3475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.12","version":"155'723","reported_seq":21598,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:06.015301+0000","last_change":"2024-09-21T23:18:05.800105+0000","last_active":"2024-09-21T23:18:06.015301+0000","last_peered":"2024-09-21T23:18:06.015301+0000","last_clean":"2024-09-21T23:18:06.015301+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:18:06.015301+0000","last_undegraded":"2024-09-21T23:18:06.015301+0000","last_fullsized":"2024-09-21T23:18:06.015301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:18.023014+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048034480000000001,"stat_sum":{"num_bytes":25840231,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":24184,"num_read_kb":46385,"num_write":711,"num_write_kb":3534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.13","version":"153'443","reported_seq":9574,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.774424+0000","last_change":"2024-09-21T23:18:04.775119+0000","last_active":"2024-09-21T23:18:04.774424+0000","last_peered":"2024-09-21T23:18:04.774424+0000","last_clean":"2024-09-21T23:18:04.774424+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:18:04.774424+0000","last_undegraded":"2024-09-21T23:18:04.774424+0000","last_fullsized":"2024-09-21T23:18:04.774424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:48:04.812756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022510160000000002,"stat_sum":{"num_bytes":11851374,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":9378,"num_read_kb":22472,"num_write":502,"num_write_kb":2234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":602316534,"num_objects":1281,"num_object_clones":323,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1281,"num_whiteouts":615,"num_read":306559,"num_read_kb":1073114,"num_write":20018,"num_write_kb":207881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":67993600,"data_stored":77764807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":18301,"ondisk_log_size":18301,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66376,"kb_used_data":21300,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94305464,"statfs":{"total":96636764160,"available":96568795136,"internally_reserved":0,"allocated":21811200,"data_stored":24286150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607596,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67564,"kb_used_data":18316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94304276,"statfs":{"total":96636764160,"available":96567578624,"internally_reserved":0,"allocated":18755584,"data_stored":20921792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89888,"kb_used_data":18444,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94281952,"statfs":{"total":96636764160,"available":96544718848,"internally_reserved":0,"allocated":18886656,"data_stored":19400679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005082,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70920,"kb_used_data":16360,"kb_used_omap":5,"kb_used_meta":54522,"kb_avail":94300920,"statfs":{"total":96636764160,"available":96564142080,"internally_reserved":0,"allocated":16752640,"data_stored":17099062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":55830821},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14184448,"data_stored":15616854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16461824,"data_stored":18052938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16924672,"data_stored":20164422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19963904,"data_stored":23523377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:10.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:10.303 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:10.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:10.304 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:10.305 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:10.305 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:10.305 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:10.305 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:10.305 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:10.306 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:10.302+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:10.316 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:10.317 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:10.401 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:10.396+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:10.435 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:10.435 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:10.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:10.501 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:10.502+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:10.569 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:10.569 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:10.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:10.617 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:10.618 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:10.618+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:10.636 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-21T23:18:09.801201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602906921,"num_objects":1285,"num_object_clones":323,"num_object_copies":2570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1285,"num_whiteouts":615,"num_read":306675,"num_read_kb":1073214,"num_write":20179,"num_write_kb":210299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2134,"num_bytes_recovered":24352285,"num_keys_recovered":317,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18421,"ondisk_log_size":18421,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":294748,"kb_used_data":74420,"kb_used_omap":10,"kb_used_meta":220213,"kb_avail":377192612,"statfs":{"total":386547056640,"available":386245234688,"internally_reserved":0,"allocated":76206080,"data_stored":81707683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":225498740},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47702190,"num_objects":-57,"num_object_clones":-25,"num_object_copies":-114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-57,"num_whiteouts":-18,"num_read":152,"num_read_kb":126,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025478"},"pg_stats":[{"pgid":"3.1f","version":"152'518","reported_seq":13187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.805444+0000","last_change":"2024-09-21T23:18:05.805444+0000","last_active":"2024-09-21T23:18:05.805444+0000","last_peered":"2024-09-21T23:18:05.805444+0000","last_clean":"2024-09-21T23:18:05.805444+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:18:05.805444+0000","last_undegraded":"2024-09-21T23:18:05.805444+0000","last_fullsized":"2024-09-21T23:18:05.805444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:19:10.321436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073146249999999999,"stat_sum":{"num_bytes":27626073,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":13294,"num_read_kb":21607,"num_write":587,"num_write_kb":3598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.1e","version":"152'447","reported_seq":5449,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838461+0000","last_change":"2024-09-21T23:18:04.838712+0000","last_active":"2024-09-21T23:18:04.838461+0000","last_peered":"2024-09-21T23:18:04.838461+0000","last_clean":"2024-09-21T23:18:04.838461+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:18:04.838461+0000","last_undegraded":"2024-09-21T23:18:04.838461+0000","last_fullsized":"2024-09-21T23:18:04.838461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:30:26.581188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093906199999999995,"stat_sum":{"num_bytes":22345283,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":5561,"num_read_kb":31938,"num_write":406,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1d","version":"153'569","reported_seq":7847,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.773961+0000","last_change":"2024-09-21T23:18:04.774395+0000","last_active":"2024-09-21T23:18:04.773961+0000","last_peered":"2024-09-21T23:18:04.773961+0000","last_clean":"2024-09-21T23:18:04.773961+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:04.773961+0000","last_undegraded":"2024-09-21T23:18:04.773961+0000","last_fullsized":"2024-09-21T23:18:04.773961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:00:14.088875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015720549999999999,"stat_sum":{"num_bytes":15382528,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":18,"num_read":8718,"num_read_kb":41550,"num_write":589,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1c","version":"154'901","reported_seq":12405,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.889490+0000","last_change":"2024-09-21T23:18:04.777771+0000","last_active":"2024-09-21T23:18:04.889490+0000","last_peered":"2024-09-21T23:18:04.889490+0000","last_clean":"2024-09-21T23:18:04.889490+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:04.889490+0000","last_undegraded":"2024-09-21T23:18:04.889490+0000","last_fullsized":"2024-09-21T23:18:04.889490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:51:38.857267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047225660000000001,"stat_sum":{"num_bytes":25558608,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":17022,"num_read_kb":70508,"num_write":948,"num_write_kb":18787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.1b","version":"153'435","reported_seq":4981,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.795554+0000","last_change":"2024-09-21T23:18:05.795932+0000","last_active":"2024-09-21T23:18:05.795554+0000","last_peered":"2024-09-21T23:18:05.795554+0000","last_clean":"2024-09-21T23:18:05.795554+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:05.795554+0000","last_undegraded":"2024-09-21T23:18:05.795554+0000","last_fullsized":"2024-09-21T23:18:05.795554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:53:53.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670577,"stat_sum":{"num_bytes":11004483,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":5703,"num_read_kb":22515,"num_write":422,"num_write_kb":1615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.1a","version":"153'641","reported_seq":9952,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.817694+0000","last_change":"2024-09-21T23:18:04.783266+0000","last_active":"2024-09-21T23:18:04.817694+0000","last_peered":"2024-09-21T23:18:04.817694+0000","last_clean":"2024-09-21T23:18:04.817694+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:04.817694+0000","last_undegraded":"2024-09-21T23:18:04.817694+0000","last_fullsized":"2024-09-21T23:18:04.817694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:33.516022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001320321,"stat_sum":{"num_bytes":8963107,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":11645,"num_read_kb":26350,"num_write":663,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.19","version":"152'380","reported_seq":5192,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837483+0000","last_change":"2024-09-21T23:18:04.837992+0000","last_active":"2024-09-21T23:18:04.837483+0000","last_peered":"2024-09-21T23:18:04.837483+0000","last_clean":"2024-09-21T23:18:04.837483+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:04.837483+0000","last_undegraded":"2024-09-21T23:18:04.837483+0000","last_fullsized":"2024-09-21T23:18:04.837483+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:37:46.052666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074324199999999997,"stat_sum":{"num_bytes":14995504,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":4913,"num_read_kb":66320,"num_write":312,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.18","version":"155'595","reported_seq":18302,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:06.020268+0000","last_change":"2024-09-21T23:18:05.801264+0000","last_active":"2024-09-21T23:18:06.020268+0000","last_peered":"2024-09-21T23:18:06.020268+0000","last_clean":"2024-09-21T23:18:06.020268+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:18:06.020268+0000","last_undegraded":"2024-09-21T23:18:06.020268+0000","last_fullsized":"2024-09-21T23:18:06.020268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:22:42.271596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052176990000000001,"stat_sum":{"num_bytes":43998796,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":16650,"num_read_kb":62664,"num_write":730,"num_write_kb":13737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.17","version":"155'382","reported_seq":13429,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.807148+0000","last_change":"2024-09-21T23:18:05.807148+0000","last_active":"2024-09-21T23:18:05.807148+0000","last_peered":"2024-09-21T23:18:05.807148+0000","last_clean":"2024-09-21T23:18:05.807148+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:05.807148+0000","last_undegraded":"2024-09-21T23:18:05.807148+0000","last_fullsized":"2024-09-21T23:18:05.807148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:31.915635+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0093331360000000006,"stat_sum":{"num_bytes":19626048,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":13363,"num_read_kb":35456,"num_write":325,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.16","version":"153'508","reported_seq":5684,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.775263+0000","last_change":"2024-09-21T23:18:04.775964+0000","last_active":"2024-09-21T23:18:04.775263+0000","last_peered":"2024-09-21T23:18:04.775263+0000","last_clean":"2024-09-21T23:18:04.775263+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:18:04.775263+0000","last_undegraded":"2024-09-21T23:18:04.775263+0000","last_fullsized":"2024-09-21T23:18:04.775263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:55.776886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028108439999999998,"stat_sum":{"num_bytes":44221602,"num_objects":76,"num_object_clones":27,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":41,"num_read":5526,"num_read_kb":23457,"num_write":567,"num_write_kb":3547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.15","version":"154'335","reported_seq":4069,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.804223+0000","last_change":"2024-09-21T23:18:05.804991+0000","last_active":"2024-09-21T23:18:05.804223+0000","last_peered":"2024-09-21T23:18:05.804223+0000","last_clean":"2024-09-21T23:18:05.804223+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:18:05.804223+0000","last_undegraded":"2024-09-21T23:18:05.804223+0000","last_fullsized":"2024-09-21T23:18:05.804223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:26:22.014694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071129269999999998,"stat_sum":{"num_bytes":20287488,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":3872,"num_read_kb":30392,"num_write":382,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.14","version":"151'853","reported_seq":7922,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.784458+0000","last_change":"2024-09-21T23:18:04.785083+0000","last_active":"2024-09-21T23:18:04.784458+0000","last_peered":"2024-09-21T23:18:04.784458+0000","last_clean":"2024-09-21T23:18:04.784458+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:18:04.784458+0000","last_undegraded":"2024-09-21T23:18:04.784458+0000","last_fullsized":"2024-09-21T23:18:04.784458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:44.895291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020148029999999999,"stat_sum":{"num_bytes":14212255,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":11159,"num_read_kb":36984,"num_write":1027,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":273,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.783375+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:18:04.783375+0000","last_peered":"2024-09-21T23:18:04.783375+0000","last_clean":"2024-09-21T23:18:04.783375+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:18:04.783375+0000","last_undegraded":"2024-09-21T23:18:04.783375+0000","last_fullsized":"2024-09-21T23:18:04.783375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"155'702","reported_seq":20008,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.804140+0000","last_change":"2024-09-21T23:18:05.804140+0000","last_active":"2024-09-21T23:18:05.804140+0000","last_peered":"2024-09-21T23:18:05.804140+0000","last_clean":"2024-09-21T23:18:05.804140+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:05.804140+0000","last_undegraded":"2024-09-21T23:18:05.804140+0000","last_fullsized":"2024-09-21T23:18:05.804140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:47:16.366244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0064937900000000002,"stat_sum":{"num_bytes":27026515,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":20805,"num_read_kb":33863,"num_write":997,"num_write_kb":4689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":176,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837862+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:18:04.837862+0000","last_peered":"2024-09-21T23:18:04.837862+0000","last_clean":"2024-09-21T23:18:04.837862+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:18:04.837862+0000","last_undegraded":"2024-09-21T23:18:04.837862+0000","last_fullsized":"2024-09-21T23:18:04.837862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"153'1614","reported_seq":8335,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838317+0000","last_change":"2024-09-21T23:18:04.838581+0000","last_active":"2024-09-21T23:18:04.838317+0000","last_peered":"2024-09-21T23:18:04.838317+0000","last_clean":"2024-09-21T23:18:04.838317+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:18:04.838317+0000","last_undegraded":"2024-09-21T23:18:04.838317+0000","last_fullsized":"2024-09-21T23:18:04.838317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1614,"log_dups_size":0,"ondisk_log_size":1614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:17:29.861515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089964299999999999,"stat_sum":{"num_bytes":11518890,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":13322,"num_read_kb":49388,"num_write":2770,"num_write_kb":8366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.797736+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:18:05.797736+0000","last_peered":"2024-09-21T23:18:05.797736+0000","last_clean":"2024-09-21T23:18:05.797736+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:18:05.797736+0000","last_undegraded":"2024-09-21T23:18:05.797736+0000","last_fullsized":"2024-09-21T23:18:05.797736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"151'404","reported_seq":2561,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837422+0000","last_change":"2024-09-21T23:18:04.837774+0000","last_active":"2024-09-21T23:18:04.837422+0000","last_peered":"2024-09-21T23:18:04.837422+0000","last_clean":"2024-09-21T23:18:04.837422+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:18:04.837422+0000","last_undegraded":"2024-09-21T23:18:04.837422+0000","last_fullsized":"2024-09-21T23:18:04.837422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:59:29.679115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055546199999999995,"stat_sum":{"num_bytes":8433438,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":2773,"num_read_kb":10789,"num_write":433,"num_write_kb":1935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837075+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:04.837075+0000","last_peered":"2024-09-21T23:18:04.837075+0000","last_clean":"2024-09-21T23:18:04.837075+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:04.837075+0000","last_undegraded":"2024-09-21T23:18:04.837075+0000","last_fullsized":"2024-09-21T23:18:04.837075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"153'478","reported_seq":10360,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.778031+0000","last_change":"2024-09-21T23:18:04.778760+0000","last_active":"2024-09-21T23:18:04.778031+0000","last_peered":"2024-09-21T23:18:04.778031+0000","last_clean":"2024-09-21T23:18:04.778031+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:18:04.778031+0000","last_undegraded":"2024-09-21T23:18:04.778031+0000","last_fullsized":"2024-09-21T23:18:04.778031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:04.376759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050390970000000002,"stat_sum":{"num_bytes":37486789,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":11101,"num_read_kb":76330,"num_write":579,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.3","version":"154'630","reported_seq":6550,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.904088+0000","last_change":"2024-09-21T23:18:04.784661+0000","last_active":"2024-09-21T23:18:04.904088+0000","last_peered":"2024-09-21T23:18:04.904088+0000","last_clean":"2024-09-21T23:18:04.904088+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:04.904088+0000","last_undegraded":"2024-09-21T23:18:04.904088+0000","last_fullsized":"2024-09-21T23:18:04.904088+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:05:38.872903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001617786,"stat_sum":{"num_bytes":13123137,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":8001,"num_read_kb":28761,"num_write":625,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.2","version":"19'2","reported_seq":312,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.772295+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:04.772295+0000","last_peered":"2024-09-21T23:18:04.772295+0000","last_clean":"2024-09-21T23:18:04.772295+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:04.772295+0000","last_undegraded":"2024-09-21T23:18:04.772295+0000","last_fullsized":"2024-09-21T23:18:04.772295+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"153'320","reported_seq":6089,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838662+0000","last_change":"2024-09-21T23:18:04.838908+0000","last_active":"2024-09-21T23:18:04.838662+0000","last_peered":"2024-09-21T23:18:04.838662+0000","last_clean":"2024-09-21T23:18:04.838662+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:18:04.838662+0000","last_undegraded":"2024-09-21T23:18:04.838662+0000","last_fullsized":"2024-09-21T23:18:04.838662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:23.240446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061421999999999996,"stat_sum":{"num_bytes":14368372,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":5593,"num_read_kb":23746,"num_write":279,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":277,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837085+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:18:04.837085+0000","last_peered":"2024-09-21T23:18:04.837085+0000","last_clean":"2024-09-21T23:18:04.837085+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:18:04.837085+0000","last_undegraded":"2024-09-21T23:18:04.837085+0000","last_fullsized":"2024-09-21T23:18:04.837085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"155'379","reported_seq":4535,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.803347+0000","last_change":"2024-09-21T23:18:05.803347+0000","last_active":"2024-09-21T23:18:05.803347+0000","last_peered":"2024-09-21T23:18:05.803347+0000","last_clean":"2024-09-21T23:18:05.803347+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:05.803347+0000","last_undegraded":"2024-09-21T23:18:05.803347+0000","last_fullsized":"2024-09-21T23:18:05.803347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:54.241364+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079002099999999995,"stat_sum":{"num_bytes":12197964,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":3901,"num_read_kb":15449,"num_write":357,"num_write_kb":5914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837758+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:18:04.837758+0000","last_peered":"2024-09-21T23:18:04.837758+0000","last_clean":"2024-09-21T23:18:04.837758+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:18:04.837758+0000","last_undegraded":"2024-09-21T23:18:04.837758+0000","last_fullsized":"2024-09-21T23:18:04.837758+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"153'589","reported_seq":16660,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.837956+0000","last_change":"2024-09-21T23:18:04.838211+0000","last_active":"2024-09-21T23:18:04.837956+0000","last_peered":"2024-09-21T23:18:04.837956+0000","last_clean":"2024-09-21T23:18:04.837956+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:04.837956+0000","last_undegraded":"2024-09-21T23:18:04.837956+0000","last_fullsized":"2024-09-21T23:18:04.837956+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:47.575390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071451299999999995,"stat_sum":{"num_bytes":20596400,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":26,"num_read":18573,"num_read_kb":53288,"num_write":489,"num_write_kb":4973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"2.3","version":"17'1","reported_seq":299,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.773238+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:18:04.773238+0000","last_peered":"2024-09-21T23:18:04.773238+0000","last_clean":"2024-09-21T23:18:04.773238+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:18:04.773238+0000","last_undegraded":"2024-09-21T23:18:04.773238+0000","last_fullsized":"2024-09-21T23:18:04.773238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":391,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.795824+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:18:05.795824+0000","last_peered":"2024-09-21T23:18:05.795824+0000","last_clean":"2024-09-21T23:18:05.795824+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:18:05.795824+0000","last_undegraded":"2024-09-21T23:18:05.795824+0000","last_fullsized":"2024-09-21T23:18:05.795824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"153'857","reported_seq":12581,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.838128+0000","last_change":"2024-09-21T23:18:04.838398+0000","last_active":"2024-09-21T23:18:04.838128+0000","last_peered":"2024-09-21T23:18:04.838128+0000","last_clean":"2024-09-21T23:18:04.838128+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:18:04.838128+0000","last_undegraded":"2024-09-21T23:18:04.838128+0000","last_fullsized":"2024-09-21T23:18:04.838128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:10.353282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084337699999999997,"stat_sum":{"num_bytes":30689824,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":13410,"num_read_kb":34760,"num_write":746,"num_write_kb":4277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.9","version":"153'487","reported_seq":4555,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.783670+0000","last_change":"2024-09-21T23:18:04.784322+0000","last_active":"2024-09-21T23:18:04.783670+0000","last_peered":"2024-09-21T23:18:04.783670+0000","last_clean":"2024-09-21T23:18:04.783670+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:18:04.783670+0000","last_undegraded":"2024-09-21T23:18:04.783670+0000","last_fullsized":"2024-09-21T23:18:04.783670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:38:54.406022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019839620000000001,"stat_sum":{"num_bytes":14514903,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":5596,"num_read_kb":14366,"num_write":396,"num_write_kb":2871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.a","version":"153'309","reported_seq":4011,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.783513+0000","last_change":"2024-09-21T23:18:04.783975+0000","last_active":"2024-09-21T23:18:04.783513+0000","last_peered":"2024-09-21T23:18:04.783513+0000","last_clean":"2024-09-21T23:18:04.783513+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:18:04.783513+0000","last_undegraded":"2024-09-21T23:18:04.783513+0000","last_fullsized":"2024-09-21T23:18:04.783513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:40:37.823983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020053599999999999,"stat_sum":{"num_bytes":13107216,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":3301,"num_read_kb":18163,"num_write":271,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.b","version":"153'695","reported_seq":8055,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.800218+0000","last_change":"2024-09-21T23:18:05.800827+0000","last_active":"2024-09-21T23:18:05.800218+0000","last_peered":"2024-09-21T23:18:05.800218+0000","last_clean":"2024-09-21T23:18:05.800218+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:05.800218+0000","last_undegraded":"2024-09-21T23:18:05.800218+0000","last_fullsized":"2024-09-21T23:18:05.800218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:31:04.326971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051601010000000003,"stat_sum":{"num_bytes":13150751,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":10071,"num_read_kb":21860,"num_write":636,"num_write_kb":2105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.c","version":"153'602","reported_seq":5693,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.777002+0000","last_change":"2024-09-21T23:18:04.777781+0000","last_active":"2024-09-21T23:18:04.777002+0000","last_peered":"2024-09-21T23:18:04.777002+0000","last_clean":"2024-09-21T23:18:04.777002+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:18:04.777002+0000","last_undegraded":"2024-09-21T23:18:04.777002+0000","last_fullsized":"2024-09-21T23:18:04.777002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:53:40.444767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043752310000000003,"stat_sum":{"num_bytes":18473517,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":7505,"num_read_kb":17833,"num_write":458,"num_write_kb":3340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.d","version":"147'440","reported_seq":5150,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.785006+0000","last_change":"2024-09-21T23:18:04.785717+0000","last_active":"2024-09-21T23:18:04.785006+0000","last_peered":"2024-09-21T23:18:04.785006+0000","last_clean":"2024-09-21T23:18:04.785006+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:18:04.785006+0000","last_undegraded":"2024-09-21T23:18:04.785006+0000","last_fullsized":"2024-09-21T23:18:04.785006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:25:06.653816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002095619,"stat_sum":{"num_bytes":7152396,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":6155,"num_read_kb":29381,"num_write":513,"num_write_kb":1707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.e","version":"154'410","reported_seq":4238,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.776859+0000","last_change":"2024-09-21T23:18:04.776859+0000","last_active":"2024-09-21T23:18:04.776859+0000","last_peered":"2024-09-21T23:18:04.776859+0000","last_clean":"2024-09-21T23:18:04.776859+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:04.776859+0000","last_undegraded":"2024-09-21T23:18:04.776859+0000","last_fullsized":"2024-09-21T23:18:04.776859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:54:29.165449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035514320000000002,"stat_sum":{"num_bytes":9538126,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":5265,"num_read_kb":14673,"num_write":376,"num_write_kb":2506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.f","version":"153'410","reported_seq":6205,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.778180+0000","last_change":"2024-09-21T23:18:04.778752+0000","last_active":"2024-09-21T23:18:04.778180+0000","last_peered":"2024-09-21T23:18:04.778180+0000","last_clean":"2024-09-21T23:18:04.778180+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:18:04.778180+0000","last_undegraded":"2024-09-21T23:18:04.778180+0000","last_fullsized":"2024-09-21T23:18:04.778180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:05.577891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004735052,"stat_sum":{"num_bytes":13897780,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":7493,"num_read_kb":34426,"num_write":376,"num_write_kb":4961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.10","version":"153'747","reported_seq":5949,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.773356+0000","last_change":"2024-09-21T23:18:04.774281+0000","last_active":"2024-09-21T23:18:04.773356+0000","last_peered":"2024-09-21T23:18:04.773356+0000","last_clean":"2024-09-21T23:18:04.773356+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:04.773356+0000","last_undegraded":"2024-09-21T23:18:04.773356+0000","last_fullsized":"2024-09-21T23:18:04.773356+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:01:05.646725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001506082,"stat_sum":{"num_bytes":14405691,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":7081,"num_read_kb":28619,"num_write":914,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.11","version":"153'498","reported_seq":5335,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.782342+0000","last_change":"2024-09-21T23:18:04.783135+0000","last_active":"2024-09-21T23:18:04.782342+0000","last_peered":"2024-09-21T23:18:04.782342+0000","last_clean":"2024-09-21T23:18:04.782342+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:18:04.782342+0000","last_undegraded":"2024-09-21T23:18:04.782342+0000","last_fullsized":"2024-09-21T23:18:04.782342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:29.093747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262399,"stat_sum":{"num_bytes":16721445,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":5625,"num_read_kb":28821,"num_write":632,"num_write_kb":3475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"a"}]},{"pgid":"3.12","version":"155'723","reported_seq":21598,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:06.015301+0000","last_change":"2024-09-21T23:18:05.800105+0000","last_active":"2024-09-21T23:18:06.015301+0000","last_peered":"2024-09-21T23:18:06.015301+0000","last_clean":"2024-09-21T23:18:06.015301+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:18:06.015301+0000","last_undegraded":"2024-09-21T23:18:06.015301+0000","last_fullsized":"2024-09-21T23:18:06.015301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:18.023014+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048034480000000001,"stat_sum":{"num_bytes":25840231,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":24184,"num_read_kb":46385,"num_write":711,"num_write_kb":3534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.13","version":"153'443","reported_seq":9574,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-21T23:18:04.774424+0000","last_change":"2024-09-21T23:18:04.775119+0000","last_active":"2024-09-21T23:18:04.774424+0000","last_peered":"2024-09-21T23:18:04.774424+0000","last_clean":"2024-09-21T23:18:04.774424+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:18:04.774424+0000","last_undegraded":"2024-09-21T23:18:04.774424+0000","last_fullsized":"2024-09-21T23:18:04.774424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:48:04.812756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022510160000000002,"stat_sum":{"num_bytes":11851374,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":9378,"num_read_kb":22472,"num_write":502,"num_write_kb":2234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":602316534,"num_objects":1281,"num_object_clones":323,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1281,"num_whiteouts":615,"num_read":306559,"num_read_kb":1073114,"num_write":20018,"num_write_kb":207881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":67993600,"data_stored":77764807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":18301,"ondisk_log_size":18301,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66376,"kb_used_data":21300,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94305464,"statfs":{"total":96636764160,"available":96568795136,"internally_reserved":0,"allocated":21811200,"data_stored":24286150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607596,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67564,"kb_used_data":18316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94304276,"statfs":{"total":96636764160,"available":96567578624,"internally_reserved":0,"allocated":18755584,"data_stored":20921792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89888,"kb_used_data":18444,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94281952,"statfs":{"total":96636764160,"available":96544718848,"internally_reserved":0,"allocated":18886656,"data_stored":19400679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005082,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70920,"kb_used_data":16360,"kb_used_omap":5,"kb_used_meta":54522,"kb_avail":94300920,"statfs":{"total":96636764160,"available":96564142080,"internally_reserved":0,"allocated":16752640,"data_stored":17099062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":55830821},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14184448,"data_stored":15616854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16461824,"data_stored":18052938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16924672,"data_stored":20164422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19963904,"data_stored":23523377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:10.638 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:18:10.638 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:18:10.638 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:10.703 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:10.700+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:10.735 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:10.736 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:10.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:10.803 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:10.806+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:10.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:18:10.880 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:10.880 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:10.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:10.943 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:10.946+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:11.005 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:11.004+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:11.014 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:11.014 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:11.025 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:11.025 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:11.035 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-21T23:18:10.897595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452151850,"num_objects":1221,"num_object_clones":306,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":600,"num_read":307635,"num_read_kb":1074040,"num_write":20345,"num_write_kb":210350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2134,"num_bytes_recovered":24352285,"num_keys_recovered":317,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18631,"ondisk_log_size":18631,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":288852,"kb_used_data":68440,"kb_used_omap":10,"kb_used_meta":220213,"kb_avail":377198508,"statfs":{"total":386547056640,"available":386251272192,"internally_reserved":0,"allocated":70082560,"data_stored":75526900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":225498740},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-150755071,"num_objects":-64,"num_object_clones":-17,"num_object_copies":-128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-64,"num_whiteouts":-15,"num_read":960,"num_read_kb":826,"num_write":166,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000863"},"pg_stats":[{"pgid":"3.1f","version":"152'518","reported_seq":13187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.805444+0000","last_change":"2024-09-21T23:18:05.805444+0000","last_active":"2024-09-21T23:18:05.805444+0000","last_peered":"2024-09-21T23:18:05.805444+0000","last_clean":"2024-09-21T23:18:05.805444+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:18:05.805444+0000","last_undegraded":"2024-09-21T23:18:05.805444+0000","last_fullsized":"2024-09-21T23:18:05.805444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:19:10.321436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073146249999999999,"stat_sum":{"num_bytes":27626073,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":13294,"num_read_kb":21607,"num_write":587,"num_write_kb":3598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.1e","version":"158'451","reported_seq":5490,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.808599+0000","last_change":"2024-09-21T23:18:09.808996+0000","last_active":"2024-09-21T23:18:09.808599+0000","last_peered":"2024-09-21T23:18:09.808599+0000","last_clean":"2024-09-21T23:18:09.808599+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:18:09.808599+0000","last_undegraded":"2024-09-21T23:18:09.808599+0000","last_fullsized":"2024-09-21T23:18:09.808599+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:41.626238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077758999999999999,"stat_sum":{"num_bytes":14156852,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":5564,"num_read_kb":31939,"num_write":408,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1d","version":"158'571","reported_seq":7884,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.809309+0000","last_change":"2024-09-21T23:18:09.809880+0000","last_active":"2024-09-21T23:18:09.809309+0000","last_peered":"2024-09-21T23:18:09.809309+0000","last_clean":"2024-09-21T23:18:09.809309+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:09.809309+0000","last_undegraded":"2024-09-21T23:18:09.809309+0000","last_fullsized":"2024-09-21T23:18:09.809309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:27:10.560992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010499999999996,"stat_sum":{"num_bytes":7583744,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":8718,"num_read_kb":41550,"num_write":589,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1c","version":"158'938","reported_seq":12581,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.836143+0000","last_change":"2024-09-21T23:18:09.813496+0000","last_active":"2024-09-21T23:18:09.836143+0000","last_peered":"2024-09-21T23:18:09.836143+0000","last_clean":"2024-09-21T23:18:09.836143+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:09.836143+0000","last_undegraded":"2024-09-21T23:18:09.836143+0000","last_fullsized":"2024-09-21T23:18:09.836143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:56:14.815882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003379235,"stat_sum":{"num_bytes":17970768,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":17250,"num_read_kb":70710,"num_write":978,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1b","version":"153'435","reported_seq":4981,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.795554+0000","last_change":"2024-09-21T23:18:05.795932+0000","last_active":"2024-09-21T23:18:05.795554+0000","last_peered":"2024-09-21T23:18:05.795554+0000","last_clean":"2024-09-21T23:18:05.795554+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:05.795554+0000","last_undegraded":"2024-09-21T23:18:05.795554+0000","last_fullsized":"2024-09-21T23:18:05.795554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:53:53.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000670577,"stat_sum":{"num_bytes":11004483,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":5703,"num_read_kb":22515,"num_write":422,"num_write_kb":1615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.1a","version":"156'642","reported_seq":10020,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.849522+0000","last_change":"2024-09-21T23:18:09.805118+0000","last_active":"2024-09-21T23:18:09.849522+0000","last_peered":"2024-09-21T23:18:09.849522+0000","last_clean":"2024-09-21T23:18:09.849522+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:09.849522+0000","last_undegraded":"2024-09-21T23:18:09.849522+0000","last_fullsized":"2024-09-21T23:18:09.849522+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:46:34.914748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017164529999999999,"stat_sum":{"num_bytes":8448547,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":24,"num_read":11677,"num_read_kb":26382,"num_write":664,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.19","version":"156'385","reported_seq":5233,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.806530+0000","last_change":"2024-09-21T23:18:09.807100+0000","last_active":"2024-09-21T23:18:09.806530+0000","last_peered":"2024-09-21T23:18:09.806530+0000","last_clean":"2024-09-21T23:18:09.806530+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:09.806530+0000","last_undegraded":"2024-09-21T23:18:09.806530+0000","last_fullsized":"2024-09-21T23:18:09.806530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:57:02.577668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085873999999999998,"stat_sum":{"num_bytes":10931744,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":4916,"num_read_kb":66321,"num_write":313,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.18","version":"155'595","reported_seq":18302,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:06.020268+0000","last_change":"2024-09-21T23:18:05.801264+0000","last_active":"2024-09-21T23:18:06.020268+0000","last_peered":"2024-09-21T23:18:06.020268+0000","last_clean":"2024-09-21T23:18:06.020268+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:18:06.020268+0000","last_undegraded":"2024-09-21T23:18:06.020268+0000","last_fullsized":"2024-09-21T23:18:06.020268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:22:42.271596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052176990000000001,"stat_sum":{"num_bytes":43998796,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":16650,"num_read_kb":62664,"num_write":730,"num_write_kb":13737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.17","version":"155'382","reported_seq":13429,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.807148+0000","last_change":"2024-09-21T23:18:05.807148+0000","last_active":"2024-09-21T23:18:05.807148+0000","last_peered":"2024-09-21T23:18:05.807148+0000","last_clean":"2024-09-21T23:18:05.807148+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:05.807148+0000","last_undegraded":"2024-09-21T23:18:05.807148+0000","last_fullsized":"2024-09-21T23:18:05.807148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:31.915635+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0093331360000000006,"stat_sum":{"num_bytes":19626048,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":13363,"num_read_kb":35456,"num_write":325,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.16","version":"158'512","reported_seq":5723,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.812866+0000","last_change":"2024-09-21T23:18:09.813592+0000","last_active":"2024-09-21T23:18:09.812866+0000","last_peered":"2024-09-21T23:18:09.812866+0000","last_clean":"2024-09-21T23:18:09.812866+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:18:09.812866+0000","last_undegraded":"2024-09-21T23:18:09.812866+0000","last_fullsized":"2024-09-21T23:18:09.812866+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:35:52.223551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003579222,"stat_sum":{"num_bytes":36147362,"num_objects":72,"num_object_clones":25,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":39,"num_read":5526,"num_read_kb":23457,"num_write":567,"num_write_kb":3547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.15","version":"154'335","reported_seq":4069,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.804223+0000","last_change":"2024-09-21T23:18:05.804991+0000","last_active":"2024-09-21T23:18:05.804223+0000","last_peered":"2024-09-21T23:18:05.804223+0000","last_clean":"2024-09-21T23:18:05.804223+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:18:05.804223+0000","last_undegraded":"2024-09-21T23:18:05.804223+0000","last_fullsized":"2024-09-21T23:18:05.804223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:26:22.014694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071129269999999998,"stat_sum":{"num_bytes":20287488,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":3872,"num_read_kb":30392,"num_write":382,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.14","version":"158'878","reported_seq":8053,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.805664+0000","last_change":"2024-09-21T23:18:09.806001+0000","last_active":"2024-09-21T23:18:09.805664+0000","last_peered":"2024-09-21T23:18:09.805664+0000","last_clean":"2024-09-21T23:18:09.805664+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:18:09.805664+0000","last_undegraded":"2024-09-21T23:18:09.805664+0000","last_fullsized":"2024-09-21T23:18:09.805664+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:41.982297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001822452,"stat_sum":{"num_bytes":10840735,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":11327,"num_read_kb":37130,"num_write":1045,"num_write_kb":13602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":283,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.803201+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:18:09.803201+0000","last_peered":"2024-09-21T23:18:09.803201+0000","last_clean":"2024-09-21T23:18:09.803201+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:18:09.803201+0000","last_undegraded":"2024-09-21T23:18:09.803201+0000","last_fullsized":"2024-09-21T23:18:09.803201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"155'702","reported_seq":20008,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.804140+0000","last_change":"2024-09-21T23:18:05.804140+0000","last_active":"2024-09-21T23:18:05.804140+0000","last_peered":"2024-09-21T23:18:05.804140+0000","last_clean":"2024-09-21T23:18:05.804140+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:05.804140+0000","last_undegraded":"2024-09-21T23:18:05.804140+0000","last_fullsized":"2024-09-21T23:18:05.804140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:47:16.366244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0064937900000000002,"stat_sum":{"num_bytes":27026515,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":20805,"num_read_kb":33863,"num_write":997,"num_write_kb":4689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.808388+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:18:09.808388+0000","last_peered":"2024-09-21T23:18:09.808388+0000","last_clean":"2024-09-21T23:18:09.808388+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:18:09.808388+0000","last_undegraded":"2024-09-21T23:18:09.808388+0000","last_fullsized":"2024-09-21T23:18:09.808388+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"156'1615","reported_seq":8371,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.808378+0000","last_change":"2024-09-21T23:18:09.808689+0000","last_active":"2024-09-21T23:18:09.808378+0000","last_peered":"2024-09-21T23:18:09.808378+0000","last_clean":"2024-09-21T23:18:09.808378+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:18:09.808378+0000","last_undegraded":"2024-09-21T23:18:09.808378+0000","last_fullsized":"2024-09-21T23:18:09.808378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1615,"log_dups_size":0,"ondisk_log_size":1615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:19.750203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059974699999999998,"stat_sum":{"num_bytes":9421738,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":13322,"num_read_kb":49388,"num_write":2771,"num_write_kb":8366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.797736+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:18:05.797736+0000","last_peered":"2024-09-21T23:18:05.797736+0000","last_clean":"2024-09-21T23:18:05.797736+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:18:05.797736+0000","last_undegraded":"2024-09-21T23:18:05.797736+0000","last_fullsized":"2024-09-21T23:18:05.797736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"157'409","reported_seq":2602,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.807691+0000","last_change":"2024-09-21T23:18:09.808036+0000","last_active":"2024-09-21T23:18:09.807691+0000","last_peered":"2024-09-21T23:18:09.807691+0000","last_clean":"2024-09-21T23:18:09.807691+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:18:09.807691+0000","last_undegraded":"2024-09-21T23:18:09.807691+0000","last_fullsized":"2024-09-21T23:18:09.807691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:38:26.123303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013157220000000001,"stat_sum":{"num_bytes":5063203,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":2775,"num_read_kb":10790,"num_write":436,"num_write_kb":1935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":186,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.806592+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:09.806592+0000","last_peered":"2024-09-21T23:18:09.806592+0000","last_clean":"2024-09-21T23:18:09.806592+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:09.806592+0000","last_undegraded":"2024-09-21T23:18:09.806592+0000","last_fullsized":"2024-09-21T23:18:09.806592+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'493","reported_seq":10417,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.812100+0000","last_change":"2024-09-21T23:18:09.812682+0000","last_active":"2024-09-21T23:18:09.812100+0000","last_peered":"2024-09-21T23:18:09.812100+0000","last_clean":"2024-09-21T23:18:09.812100+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:18:09.812100+0000","last_undegraded":"2024-09-21T23:18:09.812100+0000","last_fullsized":"2024-09-21T23:18:09.812100+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:16.862336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00319105,"stat_sum":{"num_bytes":12211527,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":11138,"num_read_kb":76357,"num_write":596,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.3","version":"158'683","reported_seq":6741,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.804654+0000","last_change":"2024-09-21T23:18:09.804965+0000","last_active":"2024-09-21T23:18:09.804654+0000","last_peered":"2024-09-21T23:18:09.804654+0000","last_clean":"2024-09-21T23:18:09.804654+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:09.804654+0000","last_undegraded":"2024-09-21T23:18:09.804654+0000","last_fullsized":"2024-09-21T23:18:09.804654+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:32:58.118299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015294810000000001,"stat_sum":{"num_bytes":13123102,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":19,"num_read":8267,"num_read_kb":28993,"num_write":673,"num_write_kb":5566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.2","version":"19'2","reported_seq":322,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.808955+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:09.808955+0000","last_peered":"2024-09-21T23:18:09.808955+0000","last_clean":"2024-09-21T23:18:09.808955+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:09.808955+0000","last_undegraded":"2024-09-21T23:18:09.808955+0000","last_fullsized":"2024-09-21T23:18:09.808955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"158'323","reported_seq":6127,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.807185+0000","last_change":"2024-09-21T23:18:09.807559+0000","last_active":"2024-09-21T23:18:09.807185+0000","last_peered":"2024-09-21T23:18:09.807185+0000","last_clean":"2024-09-21T23:18:09.807185+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:18:09.807185+0000","last_undegraded":"2024-09-21T23:18:09.807185+0000","last_fullsized":"2024-09-21T23:18:09.807185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:46:28.075931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010761740000000001,"stat_sum":{"num_bytes":4931188,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":5593,"num_read_kb":23746,"num_write":280,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":287,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.806122+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:18:09.806122+0000","last_peered":"2024-09-21T23:18:09.806122+0000","last_clean":"2024-09-21T23:18:09.806122+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:18:09.806122+0000","last_undegraded":"2024-09-21T23:18:09.806122+0000","last_fullsized":"2024-09-21T23:18:09.806122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"155'379","reported_seq":4535,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.803347+0000","last_change":"2024-09-21T23:18:05.803347+0000","last_active":"2024-09-21T23:18:05.803347+0000","last_peered":"2024-09-21T23:18:05.803347+0000","last_clean":"2024-09-21T23:18:05.803347+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:05.803347+0000","last_undegraded":"2024-09-21T23:18:05.803347+0000","last_fullsized":"2024-09-21T23:18:05.803347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:54.241364+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079002099999999995,"stat_sum":{"num_bytes":12197964,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":3901,"num_read_kb":15449,"num_write":357,"num_write_kb":5914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":186,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.806294+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:18:09.806294+0000","last_peered":"2024-09-21T23:18:09.806294+0000","last_clean":"2024-09-21T23:18:09.806294+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:18:09.806294+0000","last_undegraded":"2024-09-21T23:18:09.806294+0000","last_fullsized":"2024-09-21T23:18:09.806294+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"159'603","reported_seq":16769,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.894499+0000","last_change":"2024-09-21T23:18:09.808165+0000","last_active":"2024-09-21T23:18:09.894499+0000","last_peered":"2024-09-21T23:18:09.894499+0000","last_clean":"2024-09-21T23:18:09.894499+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:09.894499+0000","last_undegraded":"2024-09-21T23:18:09.894499+0000","last_fullsized":"2024-09-21T23:18:09.894499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:44.506589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087897599999999995,"stat_sum":{"num_bytes":17450672,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":18744,"num_read_kb":53436,"num_write":501,"num_write_kb":4979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.3","version":"17'1","reported_seq":309,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.809072+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:18:09.809072+0000","last_peered":"2024-09-21T23:18:09.809072+0000","last_clean":"2024-09-21T23:18:09.809072+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:18:09.809072+0000","last_undegraded":"2024-09-21T23:18:09.809072+0000","last_fullsized":"2024-09-21T23:18:09.809072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":391,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.795824+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:18:05.795824+0000","last_peered":"2024-09-21T23:18:05.795824+0000","last_clean":"2024-09-21T23:18:05.795824+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:18:05.795824+0000","last_undegraded":"2024-09-21T23:18:05.795824+0000","last_fullsized":"2024-09-21T23:18:05.795824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"154'858","reported_seq":12617,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.806620+0000","last_change":"2024-09-21T23:18:09.807258+0000","last_active":"2024-09-21T23:18:09.806620+0000","last_peered":"2024-09-21T23:18:09.806620+0000","last_clean":"2024-09-21T23:18:09.806620+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:18:09.806620+0000","last_undegraded":"2024-09-21T23:18:09.806620+0000","last_fullsized":"2024-09-21T23:18:09.806620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:53.419544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010095449999999999,"stat_sum":{"num_bytes":5524000,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":13410,"num_read_kb":34760,"num_write":747,"num_write_kb":4277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.9","version":"158'490","reported_seq":4595,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.805122+0000","last_change":"2024-09-21T23:18:09.805495+0000","last_active":"2024-09-21T23:18:09.805122+0000","last_peered":"2024-09-21T23:18:09.805122+0000","last_clean":"2024-09-21T23:18:09.805122+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:18:09.805122+0000","last_undegraded":"2024-09-21T23:18:09.805122+0000","last_fullsized":"2024-09-21T23:18:09.805122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:00.985880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001894473,"stat_sum":{"num_bytes":8000199,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":5599,"num_read_kb":14367,"num_write":397,"num_write_kb":2871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.a","version":"158'314","reported_seq":4051,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.803958+0000","last_change":"2024-09-21T23:18:09.804357+0000","last_active":"2024-09-21T23:18:09.803958+0000","last_peered":"2024-09-21T23:18:09.803958+0000","last_clean":"2024-09-21T23:18:09.803958+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:18:09.803958+0000","last_undegraded":"2024-09-21T23:18:09.803958+0000","last_fullsized":"2024-09-21T23:18:09.803958+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:50:35.391298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012185309999999999,"stat_sum":{"num_bytes":5128208,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":3303,"num_read_kb":18165,"num_write":274,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.b","version":"153'695","reported_seq":8055,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:05.800218+0000","last_change":"2024-09-21T23:18:05.800827+0000","last_active":"2024-09-21T23:18:05.800218+0000","last_peered":"2024-09-21T23:18:05.800218+0000","last_clean":"2024-09-21T23:18:05.800218+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:05.800218+0000","last_undegraded":"2024-09-21T23:18:05.800218+0000","last_fullsized":"2024-09-21T23:18:05.800218+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:31:04.326971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051601010000000003,"stat_sum":{"num_bytes":13150751,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":10071,"num_read_kb":21860,"num_write":636,"num_write_kb":2105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.c","version":"156'604","reported_seq":5730,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.814153+0000","last_change":"2024-09-21T23:18:09.814866+0000","last_active":"2024-09-21T23:18:09.814153+0000","last_peered":"2024-09-21T23:18:09.814153+0000","last_clean":"2024-09-21T23:18:09.814153+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:18:09.814153+0000","last_undegraded":"2024-09-21T23:18:09.814153+0000","last_fullsized":"2024-09-21T23:18:09.814153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:34.537447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041307080000000003,"stat_sum":{"num_bytes":14159405,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":7505,"num_read_kb":17833,"num_write":460,"num_write_kb":3340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.d","version":"147'440","reported_seq":5185,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.805291+0000","last_change":"2024-09-21T23:18:09.805617+0000","last_active":"2024-09-21T23:18:09.805291+0000","last_peered":"2024-09-21T23:18:09.805291+0000","last_clean":"2024-09-21T23:18:09.805291+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:18:09.805291+0000","last_undegraded":"2024-09-21T23:18:09.805291+0000","last_fullsized":"2024-09-21T23:18:09.805291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:40:53.953699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001681763,"stat_sum":{"num_bytes":7152396,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":6155,"num_read_kb":29381,"num_write":513,"num_write_kb":1707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.e","version":"158'411","reported_seq":4274,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.812541+0000","last_change":"2024-09-21T23:18:09.812933+0000","last_active":"2024-09-21T23:18:09.812541+0000","last_peered":"2024-09-21T23:18:09.812541+0000","last_clean":"2024-09-21T23:18:09.812541+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:09.812541+0000","last_undegraded":"2024-09-21T23:18:09.812541+0000","last_fullsized":"2024-09-21T23:18:09.812541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:20.820565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030829329999999999,"stat_sum":{"num_bytes":7965262,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":5265,"num_read_kb":14673,"num_write":377,"num_write_kb":2506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.f","version":"158'426","reported_seq":6260,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.813937+0000","last_change":"2024-09-21T23:18:09.814363+0000","last_active":"2024-09-21T23:18:09.813937+0000","last_peered":"2024-09-21T23:18:09.813937+0000","last_clean":"2024-09-21T23:18:09.813937+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:18:09.813937+0000","last_undegraded":"2024-09-21T23:18:09.813937+0000","last_fullsized":"2024-09-21T23:18:09.813937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:46:07.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041988269999999996,"stat_sum":{"num_bytes":5761552,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":7524,"num_read_kb":34450,"num_write":391,"num_write_kb":4965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.10","version":"157'749","reported_seq":5988,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.811897+0000","last_change":"2024-09-21T23:18:09.812363+0000","last_active":"2024-09-21T23:18:09.811897+0000","last_peered":"2024-09-21T23:18:09.811897+0000","last_clean":"2024-09-21T23:18:09.811897+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:09.811897+0000","last_undegraded":"2024-09-21T23:18:09.811897+0000","last_fullsized":"2024-09-21T23:18:09.811897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:13:08.187577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00300592,"stat_sum":{"num_bytes":14143531,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":7084,"num_read_kb":28620,"num_write":916,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.11","version":"158'502","reported_seq":5373,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.803867+0000","last_change":"2024-09-21T23:18:09.804583+0000","last_active":"2024-09-21T23:18:09.803867+0000","last_peered":"2024-09-21T23:18:09.803867+0000","last_clean":"2024-09-21T23:18:09.803867+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:18:09.803867+0000","last_undegraded":"2024-09-21T23:18:09.803867+0000","last_fullsized":"2024-09-21T23:18:09.803867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:10.689306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016951830000000001,"stat_sum":{"num_bytes":5056037,"num_objects":34,"num_object_clones":11,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":5625,"num_read_kb":28821,"num_write":632,"num_write_kb":3475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.12","version":"155'723","reported_seq":21598,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T23:18:06.015301+0000","last_change":"2024-09-21T23:18:05.800105+0000","last_active":"2024-09-21T23:18:06.015301+0000","last_peered":"2024-09-21T23:18:06.015301+0000","last_clean":"2024-09-21T23:18:06.015301+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:18:06.015301+0000","last_undegraded":"2024-09-21T23:18:06.015301+0000","last_fullsized":"2024-09-21T23:18:06.015301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:18.023014+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048034480000000001,"stat_sum":{"num_bytes":25840231,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":24184,"num_read_kb":46385,"num_write":711,"num_write_kb":3534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"b"}]},{"pgid":"3.13","version":"159'450","reported_seq":9618,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-21T23:18:09.864376+0000","last_change":"2024-09-21T23:18:09.810260+0000","last_active":"2024-09-21T23:18:09.864376+0000","last_peered":"2024-09-21T23:18:09.864376+0000","last_clean":"2024-09-21T23:18:09.864376+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:18:09.864376+0000","last_undegraded":"2024-09-21T23:18:09.864376+0000","last_fullsized":"2024-09-21T23:18:09.864376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:49:44.037986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010387300000000001,"stat_sum":{"num_bytes":9631342,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":9389,"num_read_kb":22480,"num_write":509,"num_write_kb":2235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":451561463,"num_objects":1217,"num_object_clones":306,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":600,"num_read":307519,"num_read_kb":1073940,"num_write":20184,"num_write_kb":207932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61702144,"data_stored":71508346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":18511,"ondisk_log_size":18511,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607597,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64380,"kb_used_data":19288,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94307460,"statfs":{"total":96636764160,"available":96570839040,"internally_reserved":0,"allocated":19750912,"data_stored":22225493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65816,"kb_used_data":16576,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94306024,"statfs":{"total":96636764160,"available":96569368576,"internally_reserved":0,"allocated":16973824,"data_stored":19143322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607597,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607597,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87736,"kb_used_data":16216,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94284104,"statfs":{"total":96636764160,"available":96546922496,"internally_reserved":0,"allocated":16605184,"data_stored":17059023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005082,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70920,"kb_used_data":16360,"kb_used_omap":5,"kb_used_meta":54522,"kb_avail":94300920,"statfs":{"total":96636764160,"available":96564142080,"internally_reserved":0,"allocated":16752640,"data_stored":17099062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":55830821},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14184448,"data_stored":15616854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14114816,"data_stored":15684188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15085568,"data_stored":18358968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":17858560,"data_stored":21441120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:11.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:11.106 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:11.104+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:11.207 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:11.210+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:11.208 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:11.208 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:11.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:11.311 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:11.314+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:11.393 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:11.394 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:11.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:11.409 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:11.408+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:11.510 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:11.510+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:11.578 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:11.578 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:11.592 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:11.635 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:11.638+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:11.712 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:11.708+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:11.759 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:11.760 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:11.761 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:18:11.783 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:11.829 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:11.828+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:11.914 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:11.914+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:11.929 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:11.929 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:11.949 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:12.015 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:12.014+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:12.116 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:12.118+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:12.125 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:12.125 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:12.140 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:12.216 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:12.212+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:12.260 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:12.260 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:12.274 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:12.318 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:12.316+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:12.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:18:12.417 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:12.416+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:12.441 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:12.441 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:12.455 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:12.495 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:12.495 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:12.511 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:12.512 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:12.517 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:12.518+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:12.528 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:12.528 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:12.546 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:12.547 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:12.566 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:12.566 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:12.587 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:12.587 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:12.619 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:12.616+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:12.619 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:12.620 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:12.634 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:12.726 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:12.724+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:12.805 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 157.84799579200001, 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:12.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 157.847728808, 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.811 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:12.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.235400263, 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:18:12.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:12.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:18:12.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.235357564, 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.235318944, 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:18:12.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:18:12.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.829 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.23528707200001, 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:18:12.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:18:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.235253848, 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:18:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:18:12.837 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.235218773, 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.838 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.235173815, 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.842 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:12.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.234721748, 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:18:12.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.234007269, 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:18:12.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:18:12.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:18:12.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:18:12.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.23396619899999, 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.855 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:18:12.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:18:12.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.23350908099999, 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.859 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.233272402, 2024-09-21T23:18:12.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.867 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.232856821, 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.232179161, 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.23168787199999, 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.876 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:18:12.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:18:12.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 116.23143097400001, 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:12.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:18:12.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:18:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 59.522239874999997, 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:18:12.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:18:12.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 59.519361173999997, 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:12.888 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:12.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:12.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:18:12.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:12.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:12.893 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:12.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:12.895 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:12.892+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:12.996 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:12.998+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:13.074 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:13.074 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:13.074 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:13.074 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:13.089 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:13.137 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:13.136+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:13.197 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:13.192+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:13.210 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:13.210 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:13.224 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:13.297 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:13.298+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:13.391 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:13.391 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:13.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:13.391 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:13.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.391 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.137969304999999, 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.112920346000003, 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.397 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:18:13.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:18:13.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:18:13.402 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.079487176000001, 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:18:13.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1238 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.406 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.108736716000003, 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054982073999999999, 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1238 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280879+0000", 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8700000000000005e-07 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280883+0000", 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.912e-06 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280894+0000", 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1275e-05 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280904+0000", 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.73e-06 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335407+0000", 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054502816000000003 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335499+0000", 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2589000000000004e-05 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335860+0000", 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036096499999999998 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1341 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.410 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.105305207999997, 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055656854999999998, 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1341 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:13.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284311+0000", 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.06e-06 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284314+0000", 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.72e-06 2024-09-21T23:18:13.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284328+0000", 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3380000000000001e-05 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284338+0000", 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0052e-05 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338191+0000", 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053852786999999999 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338273+0000", 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2021000000000006e-05 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339967+0000", 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016938350000000001 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1342 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 966656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.414 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.105265228, 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056087662000000003, 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1342 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284354+0000", 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.569e-06 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284364+0000", 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0710000000000001e-05 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284373+0000", 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7819999999999999e-06 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340014+0000", 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055641057000000001 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340135+0000", 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120698 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340437+0000", 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030229200000000001 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1343 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:13.418 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.105227245000002, 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055970117, 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1343 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9200000000000001e-07 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284392+0000", 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8240000000000001e-06 2024-09-21T23:18:13.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284404+0000", 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1584e-05 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284415+0000", 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1412e-05 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339993+0000", 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055577519999999998 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340056+0000", 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3127999999999996e-05 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340358+0000", 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000302057 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.422 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1344 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.105187880999999, 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056410920000000003, 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1344 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284428+0000", 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284432+0000", 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9060000000000004e-06 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284443+0000", 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.134e-05 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284452+0000", 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0340000000000002e-06 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340391+0000", 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055938437000000001 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340464+0000", 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3744000000000004e-05 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340838+0000", 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000373595 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1345 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.105152281000002, 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056714902999999997, 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1345 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284464+0000", 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:18:13.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284467+0000", 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.63e-06 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284479+0000", 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.17e-05 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284488+0000", 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1870000000000003e-06 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340886+0000", 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056398177000000001 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340951+0000", 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4263000000000005e-05 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341178+0000", 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022679199999999999 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.430 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1346 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.105085807999998, 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057046119999999999, 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.431 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1346 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284530+0000", 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284535+0000", 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7960000000000003e-06 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284548+0000", 2024-09-21T23:18:13.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3002e-05 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284558+0000", 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8640000000000008e-06 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341212+0000", 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056654228000000001 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341281+0000", 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9061999999999997e-05 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341575+0000", 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000294152 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1419 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 32768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.104238248000001, 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056364545000000002, 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1419 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285381+0000", 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:18:13.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285391+0000", 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.03e-05 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285400+0000", 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1959999999999997e-06 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341615+0000", 2024-09-21T23:18:13.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056215380000000002 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341674+0000", 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8403000000000003e-05 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341741+0000", 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7352000000000002e-05 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1420 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.104202235999999, 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056531351000000001, 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1420 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3799999999999997e-07 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285417+0000", 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0459999999999999e-06 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285429+0000", 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1402e-05 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285438+0000", 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2050000000000008e-06 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341791+0000", 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056353542999999999 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341861+0000", 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9338000000000006e-05 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341944+0000", 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3178999999999998e-05 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1421 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.104162537000001, 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056621478000000003, 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1421 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285453+0000", 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.066e-06 2024-09-21T23:18:13.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285457+0000", 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285468+0000", 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0974e-05 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285477+0000", 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0459999999999994e-06 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341982+0000", 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056504285000000001 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342029+0000", 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7528999999999998e-05 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342074+0000", 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4647999999999997e-05 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1424 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.101656413000001, 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054910207000000003, 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.448 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1424 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287959+0000", 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0139999999999999e-06 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287965+0000", 2024-09-21T23:18:13.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287989+0000", 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4511e-05 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288003+0000", 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3318e-05 2024-09-21T23:18:13.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342685+0000", 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054682236000000002 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342764+0000", 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9122999999999994e-05 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342869+0000", 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010467399999999999 2024-09-21T23:18:13.452 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1448 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.101130212000001, 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055210594000000002, 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1448 2024-09-21T23:18:13.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288486+0000", 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4200000000000004e-07 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288490+0000", 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8979999999999998e-06 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288501+0000", 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288509+0000", 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7499999999999992e-06 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343373+0000", 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05486365 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343441+0000", 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8119000000000003e-05 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343695+0000", 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025414099999999999 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1449 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.101094179999997, 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055644921999999999, 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.457 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1449 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4600000000000005e-07 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288525+0000", 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9519999999999996e-06 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288536+0000", 2024-09-21T23:18:13.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0862e-05 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288544+0000", 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.276e-06 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343752+0000", 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055208113000000003 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343837+0000", 2024-09-21T23:18:13.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4466000000000001e-05 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344166+0000", 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032870700000000002 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1450 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.101059028000002, 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055497638000000002, 2024-09-21T23:18:13.461 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1450 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288557+0000", 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288562+0000", 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7860000000000002e-06 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288574+0000", 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1637999999999999e-05 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288583+0000", 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8240000000000006e-06 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343733+0000", 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055150504000000003 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343785+0000", 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1805999999999998e-05 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344054+0000", 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026842000000000001 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.465 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1451 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.101020599999998, 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055892521000000001, 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1451 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288595+0000", 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288599+0000", 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1999999999999996e-06 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288611+0000", 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1506e-05 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288619+0000", 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7790000000000001e-06 2024-09-21T23:18:13.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344089+0000", 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055469472999999998 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344170+0000", 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1287999999999998e-05 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344487+0000", 2024-09-21T23:18:13.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031666499999999999 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1452 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.100984419, 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056290277999999999, 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1452 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288631+0000", 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288635+0000", 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0339999999999998e-06 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288646+0000", 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1107999999999999e-05 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288656+0000", 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3319999999999998e-06 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344531+0000", 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055875073999999997 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344606+0000", 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4838000000000004e-05 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344921+0000", 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031523599999999999 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1453 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.100948563000003, 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056627027000000003, 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:13.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1453 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:13.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288667+0000", 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288671+0000", 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.293991+0000", 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053204910000000001 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.294012+0000", 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1019999999999999e-05 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344962+0000", 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050949529 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345048+0000", 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6263999999999999e-05 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:13.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345293+0000", 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000245135 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:13.478 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:13.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:13.480 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:13.482+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:13.580 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:13.576+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:13.614 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:13.614 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:13.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:13.681 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:13.680+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:13.748 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:13.748 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:13.748 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:13.748 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:13.748 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:13.763 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:13.784 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:13.780+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:13.796 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:18:13.884 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:13.886+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:13.900 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:13.900 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:13.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:13.900 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:13.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 131.740543715, 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:13.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 127.67528895700001, 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:18:13.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:13.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.277946825, 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:13.914 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.274579136, 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:13.918 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.27046955599999, 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:18:13.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:18:13.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:13.922 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.266676635, 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:13.926 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.20276075699999, 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:13.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:18:13.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.202733956, 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.20270779099999, 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:18:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:18:13.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.20267934100001, 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:18:13.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.202588633, 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.943 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.202557054, 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:18:13.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:18:13.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.202519364, 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:18:13.951 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:18:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:18:13.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.202488974, 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:13.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:18:13.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.202445557, 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.20241093200001, 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.19915743200001, 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:18:13.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.19912284999999, 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:13.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:13.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 115.98337845099999, 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:18:13.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.374403254999997, 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:13.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:18:13.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:18:13.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:18:13.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:18:13.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:18:13.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.990 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:13.990 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:13.990 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:13.991 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:13.994+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:13.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:14.092 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:14.094+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:14.158 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:14.158 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:14.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:14.193 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:14.188+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:14.293 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:14.292+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:14.339 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:14.339 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:14.339 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:14.339 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:14.339 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:14.353 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:14.394 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:14.392+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723719368, 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:18:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72367411099999, 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723624489, 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72359479399999, 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723561866, 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:18:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72352757, 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:18:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723503064, 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:18:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72347507400001, 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:18:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:18:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:18:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72345204600001, 2024-09-21T23:18:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:18:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723430536, 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72339499500001, 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:14.535 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723367358, 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:18:14.539 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72333219399999, 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:18:14.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:14.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:18:14.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:18:14.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723297163, 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:18:14.544 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:18:14.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:18:14.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.72326177799999, 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723221596, 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:14.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:18:14.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:18:14.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723184468, 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723144393, 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:14.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:18:14.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723119849, 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:18:14.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:18:14.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.723082064, 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:14.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:14.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:18:14.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:14.575 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:14.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:14.577 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:14.576+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:14.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:18:14.678 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:14.676+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:14.709 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:14.710 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:14.725 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:14.770 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:14.771 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:14.778 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:14.776+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:14.787 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:14.788 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:14.804 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:14.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:14.825 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:14.826 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:14.830 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:18:14.856 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:14.856 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:14.864 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:14.864 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:14.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:14.879 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:14.876+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:14.888 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:14.888 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:14.979 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:14.976+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:15.014 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:15.014 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:15.030 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:15.080 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:15.082+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:15.167 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:15.167 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:15.181 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:15.176+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:15.182 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:15.282 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:15.282+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:15.354 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:15.354 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:15.369 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:15.413 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:15.412+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:15.484 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:15.486+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:15.486 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:15.486 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:15.501 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:15.594 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:15.592+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:15.669 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:15.669 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:15.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:15.727 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:15.724+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:15.786 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:15.786+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:15.848 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:15.848 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:15.867 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:15.887 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:15.890+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:15.987 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:15.990+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:15.998 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:15.998 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:16.013 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:16.037 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:18:16.038 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:18:16.038 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T23:18:16.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T23:18:16.089 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:16.084+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:16.135 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:16.135 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:16.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:16.189 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:16.190+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:16.290 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:16.288+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:16.317 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:16.317 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:16.331 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 165, 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "309803bf-3365-47ac-8bf6-365125beceb1", 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-21T23:14:32.028556+0000", 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-21T23:18:15.841730+0000", 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-21T23:16:02.149456+0000", 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-21T23:16:51.115738+0000", 2024-09-21T23:18:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5806080, 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "noscrub", 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 6, 2024-09-21T23:18:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 6, 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "squid", 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-21T23:18:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:44.926374+0000", 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-21T23:18:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "15", 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:18:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:18:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:18:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 6.059999942779541, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 6.059999942779541, 2024-09-21T23:18:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:49.598603+0000", 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:18:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-21T23:18:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "19", 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:18:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 2, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 19, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:18:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:18:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2.9900000095367432, 2024-09-21T23:18:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2.9900000095367432, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.67000001668930054, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-21T23:14:59.367341+0000", 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-21T23:18:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "is_stretch_pool": false, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-21T23:18:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "165", 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T23:18:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 42, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 165, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-21T23:18:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-21T23:18:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "score_type": "Fair distribution", 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.690000057220459, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.690000057220459, 2024-09-21T23:18:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.67000001668930054, 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "a24df615-4443-4f27-888f-d1e4e7affbb3", 2024-09-21T23:18:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 12, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 29, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 61, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 131, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 31, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-21T23:18:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1028371170 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1028371170", 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1028371170", 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1028371170", 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1028371170", 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:18:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "d6765c2a-98b2-4a2b-8d8c-f3963575eb64", 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:18:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2712042978 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/2712042978", 2024-09-21T23:18:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/2712042978", 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2712042978", 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2712042978", 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "9753272a-58a9-4366-87d7-550c49134023", 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:18:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 74, 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1166517858 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/1166517858", 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/1166517858", 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/1166517858", 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/1166517858", 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:18:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "24d987f9-f2cd-4ccd-9dc8-fea64467d92d", 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 0, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 0, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 21, 2024-09-21T23:18:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6809", 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6811", 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6814", 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6816", 2024-09-21T23:18:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6822", 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6823", 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6818", 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6821", 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1039392120 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6811/1039392120", 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6816/1039392120", 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6823/1039392120", 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6821/1039392120", 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "c1c30a49-7f26-4264-a1a9-160dc552953d", 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 87, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6808", 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6810", 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6812", 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6813", 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6819", 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6820", 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6815", 2024-09-21T23:18:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6817", 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 2764095640 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6810/2764095640", 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6813/2764095640", 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6820/2764095640", 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6817/2764095640", 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:18:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f24e9341-be4e-4be2-b180-7ebd3f705e9b", 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 12, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 99, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-21T23:18:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6800", 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6801", 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6802", 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6803", 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6806", 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6807", 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6804", 2024-09-21T23:18:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.144:6805", 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1732973431 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.144:6801/1732973431", 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.144:6803/1732973431", 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.144:6807/1732973431", 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.144:6805/1732973431", 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-21T23:18:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.468749+0000", 2024-09-21T23:18:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 31 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.044837+0000", 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:18:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 8196, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.310970+0000", 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 65536, 2024-09-21T23:18:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.241402+0000", 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.877818+0000", 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540701547738038271, 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-21T23:14:41.434052+0000", 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [ 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.b", 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: 0, 2024-09-21T23:18:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: 4 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-21T23:18:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [ 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "snaps": [ 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "begin": 17, 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "length": 11 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [ 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "snaps": [ 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "begin": 17, 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "length": 1 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-21T23:18:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:16.423 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:16.425 INFO:tasks.thrashosds.thrasher:j is {'epoch': 165, 'fsid': '309803bf-3365-47ac-8bf6-365125beceb1', 'created': '2024-09-21T23:14:32.028556+0000', 'modified': '2024-09-21T23:18:15.841730+0000', 'last_up_change': '2024-09-21T23:16:02.149456+0000', 'last_in_change': '2024-09-21T23:16:51.115738+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T23:14:44.926374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T23:14:49.598603+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T23:14:59.367341+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 165, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a24df615-4443-4f27-888f-d1e4e7affbb3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 12, 'last_clean_end': 29, 'up_from': 61, 'up_thru': 131, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1028371170}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1028371170}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1028371170}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1028371170}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1028371170}]}, 'public_addr': '172.21.15.53:6817/1028371170', 'cluster_addr': '172.21.15.53:6819/1028371170', 'heartbeat_back_addr': '172.21.15.53:6823/1028371170', 'heartbeat_front_addr': '172.21.15.53:6821/1028371170', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd6765c2a-98b2-4a2b-8d8c-f3963575eb64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2712042978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2712042978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2712042978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2712042978}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2712042978}]}, 'public_addr': '172.21.15.53:6809/2712042978', 'cluster_addr': '172.21.15.53:6811/2712042978', 'heartbeat_back_addr': '172.21.15.53:6815/2712042978', 'heartbeat_front_addr': '172.21.15.53:6813/2712042978', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9753272a-58a9-4366-87d7-550c49134023', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 1166517858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 1166517858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 1166517858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 1166517858}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 1166517858}]}, 'public_addr': '172.21.15.53:6801/1166517858', 'cluster_addr': '172.21.15.53:6803/1166517858', 'heartbeat_back_addr': '172.21.15.53:6807/1166517858', 'heartbeat_front_addr': '172.21.15.53:6805/1166517858', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '24d987f9-f2cd-4ccd-9dc8-fea64467d92d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6809', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 1039392120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6816', 'nonce': 1039392120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6823', 'nonce': 1039392120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 1039392120}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 1039392120}]}, 'public_addr': '172.21.15.144:6811/1039392120', 'cluster_addr': '172.21.15.144:6816/1039392120', 'heartbeat_back_addr': '172.21.15.144:6823/1039392120', 'heartbeat_front_addr': '172.21.15.144:6821/1039392120', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'c1c30a49-7f26-4264-a1a9-160dc552953d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6810', 'nonce': 2764095640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 2764095640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2764095640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6815', 'nonce': 2764095640}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2764095640}]}, 'public_addr': '172.21.15.144:6810/2764095640', 'cluster_addr': '172.21.15.144:6813/2764095640', 'heartbeat_back_addr': '172.21.15.144:6820/2764095640', 'heartbeat_front_addr': '172.21.15.144:6817/2764095640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f24e9341-be4e-4be2-b180-7ebd3f705e9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 1732973431}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 1732973431}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 1732973431}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 1732973431}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 1732973431}]}, 'public_addr': '172.21.15.144:6801/1732973431', 'cluster_addr': '172.21.15.144:6803/1732973431', 'heartbeat_back_addr': '172.21.15.144:6807/1732973431', 'heartbeat_front_addr': '172.21.15.144:6805/1732973431', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.468749+0000', 'dead_epoch': 31}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.044837+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 8196, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.310970+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.241402+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.877818+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T23:14:41.434052+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.b', 'osds': [0, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 17, 'length': 11}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 17, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T23:18:16.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:16.426 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:16.420+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:16.525 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:16.526+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:16.552 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:16.552 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:16.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:16.625 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:16.624+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:16.694 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:16.695 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:16.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:16.755 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:16.758+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:16.798 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:16.821 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-21T23:18:16.763268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206913229,"num_objects":746,"num_object_clones":116,"num_object_copies":1492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":746,"num_whiteouts":431,"num_read":310049,"num_read_kb":1076090,"num_write":20666,"num_write_kb":210431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2134,"num_bytes_recovered":24352285,"num_keys_recovered":317,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19358,"ondisk_log_size":19358,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":274292,"kb_used_data":49808,"kb_used_omap":10,"kb_used_meta":224373,"kb_avail":377213068,"statfs":{"total":386547056640,"available":386266181632,"internally_reserved":0,"allocated":51003392,"data_stored":55297482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":229758580},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-188618490,"num_objects":-430,"num_object_clones":-180,"num_object_copies":-860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-430,"num_whiteouts":-161,"num_read":1991,"num_read_kb":1693,"num_write":265,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.940122"},"pg_stats":[{"pgid":"3.1f","version":"165'548","reported_seq":13289,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.878194+0000","last_change":"2024-09-21T23:18:15.878919+0000","last_active":"2024-09-21T23:18:15.878194+0000","last_peered":"2024-09-21T23:18:15.878194+0000","last_clean":"2024-09-21T23:18:15.878194+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:18:15.878194+0000","last_undegraded":"2024-09-21T23:18:15.878194+0000","last_fullsized":"2024-09-21T23:18:15.878194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:36.375378+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024045858,"stat_sum":{"num_bytes":2388532,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":18,"num_read":13325,"num_read_kb":21631,"num_write":604,"num_write_kb":3602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.1e","version":"164'465","reported_seq":5539,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.851023+0000","last_change":"2024-09-21T23:18:14.851464+0000","last_active":"2024-09-21T23:18:14.851023+0000","last_peered":"2024-09-21T23:18:14.851023+0000","last_clean":"2024-09-21T23:18:14.851023+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:18:14.851023+0000","last_undegraded":"2024-09-21T23:18:14.851023+0000","last_fullsized":"2024-09-21T23:18:14.851023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:15:23.387382+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0083369129999999996,"stat_sum":{"num_bytes":8337972,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":5567,"num_read_kb":31940,"num_write":412,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1d","version":"164'588","reported_seq":7933,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.846370+0000","last_change":"2024-09-21T23:18:14.847637+0000","last_active":"2024-09-21T23:18:14.846370+0000","last_peered":"2024-09-21T23:18:14.846370+0000","last_clean":"2024-09-21T23:18:14.846370+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:14.846370+0000","last_undegraded":"2024-09-21T23:18:14.846370+0000","last_fullsized":"2024-09-21T23:18:14.846370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:47.156984+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012741791000000001,"stat_sum":{"num_bytes":4784640,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":9,"num_read":8718,"num_read_kb":41550,"num_write":590,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1c","version":"164'959","reported_seq":12727,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.865295+0000","last_change":"2024-09-21T23:18:14.847541+0000","last_active":"2024-09-21T23:18:14.865295+0000","last_peered":"2024-09-21T23:18:14.865295+0000","last_clean":"2024-09-21T23:18:14.865295+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:14.865295+0000","last_undegraded":"2024-09-21T23:18:14.865295+0000","last_fullsized":"2024-09-21T23:18:14.865295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":959,"log_dups_size":0,"ondisk_log_size":959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:25:39.461527+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01265923,"stat_sum":{"num_bytes":10842176,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":17472,"num_read_kb":70905,"num_write":992,"num_write_kb":18793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1b","version":"164'451","reported_seq":5068,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.877194+0000","last_change":"2024-09-21T23:18:15.877835+0000","last_active":"2024-09-21T23:18:15.877194+0000","last_peered":"2024-09-21T23:18:15.877194+0000","last_clean":"2024-09-21T23:18:15.877194+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:15.877194+0000","last_undegraded":"2024-09-21T23:18:15.877194+0000","last_fullsized":"2024-09-21T23:18:15.877194+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:57.813780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019883054000000001,"stat_sum":{"num_bytes":5812766,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":12,"num_read":5709,"num_read_kb":22518,"num_write":428,"num_write_kb":1615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.1a","version":"164'663","reported_seq":10126,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.872045+0000","last_change":"2024-09-21T23:18:14.865442+0000","last_active":"2024-09-21T23:18:14.872045+0000","last_peered":"2024-09-21T23:18:14.872045+0000","last_clean":"2024-09-21T23:18:14.872045+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:14.872045+0000","last_undegraded":"2024-09-21T23:18:14.872045+0000","last_fullsized":"2024-09-21T23:18:14.872045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:47:55.190375+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.022800974000000002,"stat_sum":{"num_bytes":4649507,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":11731,"num_read_kb":26436,"num_write":665,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.19","version":"164'407","reported_seq":5291,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.856703+0000","last_change":"2024-09-21T23:18:14.857189+0000","last_active":"2024-09-21T23:18:14.856703+0000","last_peered":"2024-09-21T23:18:14.856703+0000","last_clean":"2024-09-21T23:18:14.856703+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:14.856703+0000","last_undegraded":"2024-09-21T23:18:14.856703+0000","last_fullsized":"2024-09-21T23:18:14.856703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:40.509556+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013919608,"stat_sum":{"num_bytes":4866048,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":319,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.18","version":"165'649","reported_seq":18495,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.866370+0000","last_change":"2024-09-21T23:18:15.866370+0000","last_active":"2024-09-21T23:18:15.866370+0000","last_peered":"2024-09-21T23:18:15.866370+0000","last_clean":"2024-09-21T23:18:15.866370+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:18:15.866370+0000","last_undegraded":"2024-09-21T23:18:15.866370+0000","last_fullsized":"2024-09-21T23:18:15.866370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:53:07.396085+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012815979999999999,"stat_sum":{"num_bytes":6294588,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":16826,"num_read_kb":62816,"num_write":757,"num_write_kb":13743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.17","version":"165'402","reported_seq":13517,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.871269+0000","last_change":"2024-09-21T23:18:15.872799+0000","last_active":"2024-09-21T23:18:15.871269+0000","last_peered":"2024-09-21T23:18:15.871269+0000","last_clean":"2024-09-21T23:18:15.871269+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:15.871269+0000","last_undegraded":"2024-09-21T23:18:15.871269+0000","last_fullsized":"2024-09-21T23:18:15.871269+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:37:51.670581+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018354453999999999,"stat_sum":{"num_bytes":4492864,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":18,"num_read":13363,"num_read_kb":35456,"num_write":327,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.16","version":"164'547","reported_seq":5786,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.872424+0000","last_change":"2024-09-21T23:18:14.873037+0000","last_active":"2024-09-21T23:18:14.872424+0000","last_peered":"2024-09-21T23:18:14.872424+0000","last_clean":"2024-09-21T23:18:14.872424+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:18:14.872424+0000","last_undegraded":"2024-09-21T23:18:14.872424+0000","last_fullsized":"2024-09-21T23:18:14.872424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:25:06.310830+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.037285998000000001,"stat_sum":{"num_bytes":15697570,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":24,"num_read":5526,"num_read_kb":23457,"num_write":570,"num_write_kb":3547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.15","version":"161'342","reported_seq":4145,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.873052+0000","last_change":"2024-09-21T23:18:15.874847+0000","last_active":"2024-09-21T23:18:15.873052+0000","last_peered":"2024-09-21T23:18:15.873052+0000","last_clean":"2024-09-21T23:18:15.873052+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:18:15.873052+0000","last_undegraded":"2024-09-21T23:18:15.873052+0000","last_fullsized":"2024-09-21T23:18:15.873052+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:08:57.092956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020417751000000001,"stat_sum":{"num_bytes":12898304,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":385,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.14","version":"164'898","reported_seq":8104,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.867829+0000","last_change":"2024-09-21T23:18:14.869365+0000","last_active":"2024-09-21T23:18:14.867829+0000","last_peered":"2024-09-21T23:18:14.867829+0000","last_clean":"2024-09-21T23:18:14.867829+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:18:14.867829+0000","last_undegraded":"2024-09-21T23:18:14.867829+0000","last_fullsized":"2024-09-21T23:18:14.867829+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":898,"log_dups_size":0,"ondisk_log_size":898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:00.061811+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.025815482000000001,"stat_sum":{"num_bytes":7260319,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":11327,"num_read_kb":37130,"num_write":1049,"num_write_kb":13602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":293,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.843752+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:18:14.843752+0000","last_peered":"2024-09-21T23:18:14.843752+0000","last_clean":"2024-09-21T23:18:14.843752+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:18:14.843752+0000","last_undegraded":"2024-09-21T23:18:14.843752+0000","last_fullsized":"2024-09-21T23:18:14.843752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"165'741","reported_seq":20119,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.875251+0000","last_change":"2024-09-21T23:18:15.875251+0000","last_active":"2024-09-21T23:18:15.875251+0000","last_peered":"2024-09-21T23:18:15.875251+0000","last_clean":"2024-09-21T23:18:15.875251+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:15.875251+0000","last_undegraded":"2024-09-21T23:18:15.875251+0000","last_fullsized":"2024-09-21T23:18:15.875251+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:45:30.509813+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021630825999999999,"stat_sum":{"num_bytes":8793640,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":20836,"num_read_kb":33887,"num_write":1014,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.843113+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:18:14.843113+0000","last_peered":"2024-09-21T23:18:14.843113+0000","last_clean":"2024-09-21T23:18:14.843113+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:18:14.843113+0000","last_undegraded":"2024-09-21T23:18:14.843113+0000","last_fullsized":"2024-09-21T23:18:14.843113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"161'1622","reported_seq":8412,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.848123+0000","last_change":"2024-09-21T23:18:14.848665+0000","last_active":"2024-09-21T23:18:14.848123+0000","last_peered":"2024-09-21T23:18:14.848123+0000","last_clean":"2024-09-21T23:18:14.848123+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:18:14.848123+0000","last_undegraded":"2024-09-21T23:18:14.848123+0000","last_fullsized":"2024-09-21T23:18:14.848123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1622,"log_dups_size":0,"ondisk_log_size":1622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:21:35.513544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054553079999999999,"stat_sum":{"num_bytes":8070058,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":15,"num_read":13322,"num_read_kb":49388,"num_write":2772,"num_write_kb":8366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":229,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.854057+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:18:15.854057+0000","last_peered":"2024-09-21T23:18:15.854057+0000","last_clean":"2024-09-21T23:18:15.854057+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:18:15.854057+0000","last_undegraded":"2024-09-21T23:18:15.854057+0000","last_fullsized":"2024-09-21T23:18:15.854057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"164'428","reported_seq":2660,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.847664+0000","last_change":"2024-09-21T23:18:14.848022+0000","last_active":"2024-09-21T23:18:14.847664+0000","last_peered":"2024-09-21T23:18:14.847664+0000","last_clean":"2024-09-21T23:18:14.847664+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:18:14.847664+0000","last_undegraded":"2024-09-21T23:18:14.847664+0000","last_fullsized":"2024-09-21T23:18:14.847664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:48:40.695223+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051108300000000002,"stat_sum":{"num_bytes":4322816,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":17,"num_read":2806,"num_read_kb":10814,"num_write":452,"num_write_kb":1939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":196,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.842693+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:14.842693+0000","last_peered":"2024-09-21T23:18:14.842693+0000","last_clean":"2024-09-21T23:18:14.842693+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:14.842693+0000","last_undegraded":"2024-09-21T23:18:14.842693+0000","last_fullsized":"2024-09-21T23:18:14.842693+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"164'506","reported_seq":10468,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.852674+0000","last_change":"2024-09-21T23:18:14.853889+0000","last_active":"2024-09-21T23:18:14.852674+0000","last_peered":"2024-09-21T23:18:14.852674+0000","last_clean":"2024-09-21T23:18:14.852674+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:18:14.852674+0000","last_undegraded":"2024-09-21T23:18:14.852674+0000","last_fullsized":"2024-09-21T23:18:14.852674+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:47.758468+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018435562999999999,"stat_sum":{"num_bytes":10198343,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":11148,"num_read_kb":76362,"num_write":599,"num_write_kb":6561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.3","version":"163'711","reported_seq":6824,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.852315+0000","last_change":"2024-09-21T23:18:14.854490+0000","last_active":"2024-09-21T23:18:14.852315+0000","last_peered":"2024-09-21T23:18:14.852315+0000","last_clean":"2024-09-21T23:18:14.852315+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:14.852315+0000","last_undegraded":"2024-09-21T23:18:14.852315+0000","last_fullsized":"2024-09-21T23:18:14.852315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:36.313840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011883283999999999,"stat_sum":{"num_bytes":7304734,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":17,"num_read":8297,"num_read_kb":29023,"num_write":697,"num_write_kb":5581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.2","version":"19'2","reported_seq":332,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.834124+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:14.834124+0000","last_peered":"2024-09-21T23:18:14.834124+0000","last_clean":"2024-09-21T23:18:14.834124+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:14.834124+0000","last_undegraded":"2024-09-21T23:18:14.834124+0000","last_fullsized":"2024-09-21T23:18:14.834124+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'339","reported_seq":6255,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.896595+0000","last_change":"2024-09-21T23:18:14.853026+0000","last_active":"2024-09-21T23:18:14.896595+0000","last_peered":"2024-09-21T23:18:14.896595+0000","last_clean":"2024-09-21T23:18:14.896595+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:18:14.896595+0000","last_undegraded":"2024-09-21T23:18:14.896595+0000","last_fullsized":"2024-09-21T23:18:14.896595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:32:01.223345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0081814030000000003,"stat_sum":{"num_bytes":3751540,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":5832,"num_read_kb":23951,"num_write":292,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":297,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.842602+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:18:14.842602+0000","last_peered":"2024-09-21T23:18:14.842602+0000","last_clean":"2024-09-21T23:18:14.842602+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:18:14.842602+0000","last_undegraded":"2024-09-21T23:18:14.842602+0000","last_fullsized":"2024-09-21T23:18:14.842602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"164'402","reported_seq":4626,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.862780+0000","last_change":"2024-09-21T23:18:15.865024+0000","last_active":"2024-09-21T23:18:15.862780+0000","last_peered":"2024-09-21T23:18:15.862780+0000","last_clean":"2024-09-21T23:18:15.862780+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:15.862780+0000","last_undegraded":"2024-09-21T23:18:15.862780+0000","last_fullsized":"2024-09-21T23:18:15.862780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:38:14.267641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011398948000000001,"stat_sum":{"num_bytes":3952676,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":12,"num_read":3903,"num_read_kb":15450,"num_write":360,"num_write_kb":5914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":197,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.843443+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:18:14.843443+0000","last_peered":"2024-09-21T23:18:14.843443+0000","last_clean":"2024-09-21T23:18:14.843443+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:18:14.843443+0000","last_undegraded":"2024-09-21T23:18:14.843443+0000","last_fullsized":"2024-09-21T23:18:14.843443+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"162'624","reported_seq":16846,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.851645+0000","last_change":"2024-09-21T23:18:14.852380+0000","last_active":"2024-09-21T23:18:14.851645+0000","last_peered":"2024-09-21T23:18:14.851645+0000","last_clean":"2024-09-21T23:18:14.851645+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:14.851645+0000","last_undegraded":"2024-09-21T23:18:14.851645+0000","last_fullsized":"2024-09-21T23:18:14.851645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:22:42.039980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087614240000000003,"stat_sum":{"num_bytes":13215344,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":22,"num_read":18783,"num_read_kb":53470,"num_write":509,"num_write_kb":4979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.3","version":"17'1","reported_seq":319,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.834444+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:18:14.834444+0000","last_peered":"2024-09-21T23:18:14.834444+0000","last_clean":"2024-09-21T23:18:14.834444+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:18:14.834444+0000","last_undegraded":"2024-09-21T23:18:14.834444+0000","last_fullsized":"2024-09-21T23:18:14.834444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":411,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:15.853152+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:18:15.853152+0000","last_peered":"2024-09-21T23:18:15.853152+0000","last_clean":"2024-09-21T23:18:15.853152+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:18:15.853152+0000","last_undegraded":"2024-09-21T23:18:15.853152+0000","last_fullsized":"2024-09-21T23:18:15.853152+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"164'870","reported_seq":12665,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.859507+0000","last_change":"2024-09-21T23:18:14.859865+0000","last_active":"2024-09-21T23:18:14.859507+0000","last_peered":"2024-09-21T23:18:14.859507+0000","last_clean":"2024-09-21T23:18:14.859507+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:18:14.859507+0000","last_undegraded":"2024-09-21T23:18:14.859507+0000","last_fullsized":"2024-09-21T23:18:14.859507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:46.586222+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013884336000000001,"stat_sum":{"num_bytes":3641872,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":17,"num_read":13413,"num_read_kb":34761,"num_write":749,"num_write_kb":4277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.9","version":"161'514","reported_seq":4658,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.854707+0000","last_change":"2024-09-21T23:18:14.857705+0000","last_active":"2024-09-21T23:18:14.854707+0000","last_peered":"2024-09-21T23:18:14.854707+0000","last_clean":"2024-09-21T23:18:14.854707+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:18:14.854707+0000","last_undegraded":"2024-09-21T23:18:14.854707+0000","last_fullsized":"2024-09-21T23:18:14.854707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:55:24.682521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014238821,"stat_sum":{"num_bytes":5206528,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":5634,"num_read_kb":14393,"num_write":414,"num_write_kb":2875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.a","version":"164'334","reported_seq":4103,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.852994+0000","last_change":"2024-09-21T23:18:14.854162+0000","last_active":"2024-09-21T23:18:14.852994+0000","last_peered":"2024-09-21T23:18:14.852994+0000","last_clean":"2024-09-21T23:18:14.852994+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:18:14.852994+0000","last_undegraded":"2024-09-21T23:18:14.852994+0000","last_fullsized":"2024-09-21T23:18:14.852994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:49:49.328801+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012117986000000001,"stat_sum":{"num_bytes":1988624,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":13,"num_read":3307,"num_read_kb":18169,"num_write":280,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.b","version":"164'731","reported_seq":8311,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:16.071069+0000","last_change":"2024-09-21T23:18:15.877091+0000","last_active":"2024-09-21T23:18:16.071069+0000","last_peered":"2024-09-21T23:18:16.071069+0000","last_clean":"2024-09-21T23:18:16.071069+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:16.071069+0000","last_undegraded":"2024-09-21T23:18:16.071069+0000","last_fullsized":"2024-09-21T23:18:16.071069+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:33:21.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020862024999999999,"stat_sum":{"num_bytes":3923487,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":10518,"num_read_kb":22245,"num_write":657,"num_write_kb":2111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.c","version":"164'661","reported_seq":5994,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.852210+0000","last_change":"2024-09-21T23:18:14.853398+0000","last_active":"2024-09-21T23:18:14.852210+0000","last_peered":"2024-09-21T23:18:14.852210+0000","last_clean":"2024-09-21T23:18:14.852210+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:18:14.852210+0000","last_undegraded":"2024-09-21T23:18:14.852210+0000","last_fullsized":"2024-09-21T23:18:14.852210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:12.822043+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018430424000000001,"stat_sum":{"num_bytes":5425709,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":7945,"num_read_kb":18214,"num_write":497,"num_write_kb":3352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.d","version":"164'448","reported_seq":5237,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.869328+0000","last_change":"2024-09-21T23:18:14.870282+0000","last_active":"2024-09-21T23:18:14.869328+0000","last_peered":"2024-09-21T23:18:14.869328+0000","last_clean":"2024-09-21T23:18:14.869328+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:18:14.869328+0000","last_undegraded":"2024-09-21T23:18:14.869328+0000","last_fullsized":"2024-09-21T23:18:14.869328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:40:56.924656+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.026703012000000002,"stat_sum":{"num_bytes":5444218,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":6175,"num_read_kb":29391,"num_write":515,"num_write_kb":1707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.e","version":"164'433","reported_seq":4330,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.898560+0000","last_change":"2024-09-21T23:18:14.864741+0000","last_active":"2024-09-21T23:18:14.898560+0000","last_peered":"2024-09-21T23:18:14.898560+0000","last_clean":"2024-09-21T23:18:14.898560+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:14.898560+0000","last_undegraded":"2024-09-21T23:18:14.898560+0000","last_fullsized":"2024-09-21T23:18:14.898560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:06:02.434981+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.028915250999999999,"stat_sum":{"num_bytes":5050446,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":5275,"num_read_kb":14680,"num_write":382,"num_write_kb":2506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.f","version":"162'450","reported_seq":6410,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.864908+0000","last_change":"2024-09-21T23:18:14.865799+0000","last_active":"2024-09-21T23:18:14.864908+0000","last_peered":"2024-09-21T23:18:14.864908+0000","last_clean":"2024-09-21T23:18:14.864908+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:18:14.864908+0000","last_undegraded":"2024-09-21T23:18:14.864908+0000","last_fullsized":"2024-09-21T23:18:14.864908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:15:35.156193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.029836246,"stat_sum":{"num_bytes":5008400,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":7760,"num_read_kb":34654,"num_write":410,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.10","version":"164'763","reported_seq":6040,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.838332+0000","last_change":"2024-09-21T23:18:14.839322+0000","last_active":"2024-09-21T23:18:14.838332+0000","last_peered":"2024-09-21T23:18:14.838332+0000","last_clean":"2024-09-21T23:18:14.838332+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:14.838332+0000","last_undegraded":"2024-09-21T23:18:14.838332+0000","last_fullsized":"2024-09-21T23:18:14.838332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:31:35.542706+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049801159999999997,"stat_sum":{"num_bytes":5904896,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":7092,"num_read_kb":28624,"num_write":918,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.11","version":"161'514","reported_seq":5417,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.843942+0000","last_change":"2024-09-21T23:18:14.852009+0000","last_active":"2024-09-21T23:18:14.843942+0000","last_peered":"2024-09-21T23:18:14.843942+0000","last_clean":"2024-09-21T23:18:14.843942+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:18:14.843942+0000","last_undegraded":"2024-09-21T23:18:14.843942+0000","last_fullsized":"2024-09-21T23:18:14.843942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:05:42.997452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095245769999999993,"stat_sum":{"num_bytes":3282469,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":5625,"num_read_kb":28821,"num_write":632,"num_write_kb":3475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.12","version":"165'759","reported_seq":21793,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T23:18:16.096904+0000","last_change":"2024-09-21T23:18:15.862503+0000","last_active":"2024-09-21T23:18:16.096904+0000","last_peered":"2024-09-21T23:18:16.096904+0000","last_clean":"2024-09-21T23:18:16.096904+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:18:16.096904+0000","last_undegraded":"2024-09-21T23:18:16.096904+0000","last_fullsized":"2024-09-21T23:18:16.096904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:03.933814+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0089063670000000001,"stat_sum":{"num_bytes":6089276,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":24495,"num_read_kb":46643,"num_write":736,"num_write_kb":3543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.13","version":"161'469","reported_seq":9672,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-21T23:18:14.834956+0000","last_change":"2024-09-21T23:18:14.836699+0000","last_active":"2024-09-21T23:18:14.834956+0000","last_peered":"2024-09-21T23:18:14.834956+0000","last_clean":"2024-09-21T23:18:14.834956+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:18:14.834956+0000","last_undegraded":"2024-09-21T23:18:14.834956+0000","last_fullsized":"2024-09-21T23:18:14.834956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:02:48.124785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023081809999999999,"stat_sum":{"num_bytes":7422480,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":9409,"num_read_kb":22496,"num_write":522,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":206322842,"num_objects":742,"num_object_clones":116,"num_object_copies":1484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":742,"num_whiteouts":431,"num_read":309933,"num_read_kb":1075990,"num_write":20505,"num_write_kb":208013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41897984,"data_stored":50869133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":19238,"ondisk_log_size":19238,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607598,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63308,"kb_used_data":14068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308532,"statfs":{"total":96636764160,"available":96571936768,"internally_reserved":0,"allocated":14405632,"data_stored":16868840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607598,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63956,"kb_used_data":14720,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94307884,"statfs":{"total":96636764160,"available":96571273216,"internally_reserved":0,"allocated":15073280,"data_stored":17314806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607599,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83520,"kb_used_data":12068,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94288320,"statfs":{"total":96636764160,"available":96551239680,"internally_reserved":0,"allocated":12357632,"data_stored":12235531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005084,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63508,"kb_used_data":8952,"kb_used_omap":5,"kb_used_meta":54522,"kb_avail":94308332,"statfs":{"total":96636764160,"available":96571731968,"internally_reserved":0,"allocated":9166848,"data_stored":8878305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":55830821},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6418432,"data_stored":7297048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9687040,"data_stored":10758030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13008896,"data_stored":16428204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12324864,"data_stored":15978635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:16.823 INFO:tasks.thrashosds.thrasher:Setting 3.c to [1, 2] 2024-09-21T23:18:16.823 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.c', '1', '2'] 2024-09-21T23:18:16.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.c 1 2 2024-09-21T23:18:16.840 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:16.841 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:16.841 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:16.842+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:16.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:16.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:18:16.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:18:16.929 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:16.924+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:17.029 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:17.030+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:17.039 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:17.039 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:17.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:17.131 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:17.128+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:17.137 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:17.137 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:17.171 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:17.171 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:17.190 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:17.190 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:17.211 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:17.216 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:17.216 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:17.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:17.232 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:17.234+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:17.232 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:17.233 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:17.256 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:17.256 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:17.332 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:17.334+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:17.349 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:17.349 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:17.349 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:17.349 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:17.349 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:17.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:17.451 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:17.454+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.767097087, 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:18:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:18:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:18:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.76705183, 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:18:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:18:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.76700220799999, 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:18:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766972513, 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:18:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:18:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766939585, 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:18:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:18:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:18:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.76690528899999, 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:18:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:18:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766880783, 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:18:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:18:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766852793, 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:18:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766829765, 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766808255, 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:18:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:18:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:18:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766772714, 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:18:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:18:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766745077, 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766709913, 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:18:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:18:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766674882, 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:18:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:18:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:18:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766639497, 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.76659931499999, 2024-09-21T23:18:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:18:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:18:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:18:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.76656218700001, 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:18:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766522112, 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:18:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:18:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.76649756800001, 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:18:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:18:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 120.766459783, 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:18:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:18:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:17.617 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:17.618 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:17.619 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:17.616+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:17.720 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:17.716+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:17.791 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:17.791 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:17.791 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:17.791 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:17.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:17.851 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:17.854+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:17.856 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:18:17.856 INFO:teuthology.orchestra.run.smithi053.stderr:set 3.c pg_upmap mapping to [1,2] 2024-09-21T23:18:17.927 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:17.930+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:17.934 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:17.935 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:17.935 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:17.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:18.022 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:18.020+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:18.123 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:18.120+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:18.124 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19661588, 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19559503299999, 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19553536000001, 2024-09-21T23:18:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:18:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19544538400001, 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:18:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:18:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19539547400001, 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:18:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:18:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19533933400001, 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:18:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:18:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19528555100001, 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:18:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:18:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.195253275, 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:18:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19522299499999, 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19519333900001, 2024-09-21T23:18:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:18:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.195163372, 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:18:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.195085543, 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:18:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19503783600001, 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:18:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19500764, 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:18:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19497772299999, 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:18:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:18:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:18:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.194947923, 2024-09-21T23:18:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19491798999999, 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19485302699999, 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.19482255099999, 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:18:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:18:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 163.194793387, 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:18.227 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:18.228 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:18.228 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:18.224+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:18.329 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:18.330+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:18.358 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:18.358 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:18.358 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:18.359 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:18.373 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:18.429 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:18.428+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:18.494 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:18.494 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:18.494 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:18.494 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:18.494 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:18.508 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:18.550 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:18.548+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:18.631 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:18.628+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:18.674 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 212.453975012, 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.675 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:18:18.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:18:18.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.681 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.94156972900001, 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.682 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:18:18.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:18:18.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.941504826, 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.688 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:18:18.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:18:18.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.94135838700001, 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.694 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:18.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:18:18.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.94128927200001, 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:18:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:18:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.94067694699999, 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:18:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:18:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.94047254200001, 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:18:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.94042975599999, 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:18:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54947146800001, 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54938495799999, 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:18:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54930806900001, 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.549103229, 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:18:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54901494699999, 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:18:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:18:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54888411900001, 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54883846600001, 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:18:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54880003900001, 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:18:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:18:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.548670577, 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:18:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54864893499999, 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:18:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:18:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54862889099999, 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:18:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 211.54854489900001, 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:18:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.793 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:18.795 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:18.795 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:18.792+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:18.895 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:18.898+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:28756 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=28b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e168)", 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:18:18.978495+0000", 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.000104257, 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145267, 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 28756 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978495+0000", 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978495+0000", 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978496+0000", 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.093e-06 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978502+0000", 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5979999999999996e-06 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978517+0000", 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5843999999999999e-05 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978525+0000", 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9370000000000004e-06 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978590+0000", 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509000000000002e-05 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:18.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:18:18.978651+0000", 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0733999999999998e-05 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-21T23:18:18.983 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:18.996 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:18.996 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:18.992+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:19.096 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:19.092+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:19.168 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:19.168 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:19.168 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:19.168 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:19.169 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:19.183 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:19.197 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:19.198+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:19.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:18:19.298 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:19.296+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 164.39568849700001, 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:18:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:18:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:18:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:18:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 164.395421513, 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:18:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:18:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.78309296800001, 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:18:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:18:19.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.783050269, 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.369 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:18:19.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.783011649, 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:18:19.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:18:19.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:18:19.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.78297977699999, 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.378 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:18:19.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:18:19.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.782946553, 2024-09-21T23:18:19.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:18:19.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:18:19.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.782911478, 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.387 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:18:19.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.78286652, 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.391 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:19.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:18:19.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.395 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.782414453, 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:19.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:18:19.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.78169997400001, 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:19.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.404 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.781658904, 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:18:19.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.781201786, 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:19.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:18:19.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:18:19.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.780965107, 2024-09-21T23:18:19.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:18:19.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.417 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.780549526, 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:18:19.421 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.77987186599999, 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:18:19.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.779380577, 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:18:19.426 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:18:19.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:18:19.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.77912367899999, 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:18:19.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:18:19.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:18:19.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.06993258, 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:18:19.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:18:19.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:19.439 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.067053878999999, 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:19.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:18:19.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:19.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:19.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:19.445 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:19.446 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:19.447 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:19.446+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:19.447 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:19.449 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:19.467 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:19.467 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:19.490 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:19.490 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:19.510 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:19.511 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:19.531 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:19.531 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:19.547 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:19.550+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:19.616 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:19.617 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:19.631 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:19.677 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:19.676+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:19.749 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:19.744+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:19.753 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:19.753 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:19.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:19.849 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:19.850+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:19.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:18:19.942 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:19.942 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:19.949 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:19.950+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:19.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:20.050 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:20.048+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:20.087 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:20.087 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:20.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:20.151 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:20.154+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:20.222 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:20.223 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:20.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:20.252 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:20.248+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:20.353 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:20.354+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:20.403 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:20.404 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:20.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:20.463 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:20.466+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:20.555 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:20.552+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:20.592 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:20.592 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:20.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:20.655 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:20.658+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:20.725 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:20.726 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:20.740 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:20.783 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:20.786+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:20.857 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:20.852+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:20.874 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:18:20.919 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:20.919 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:20.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:20.975 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:20.978+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:21.059 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:21.056+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:21.112 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:21.112 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:21.127 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:21.171 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:21.174+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:21.253 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:21.254 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:21.261 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:21.256+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:21.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:21.361 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:21.362+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:21.432 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:21.432 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:21.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:21.494 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:21.492+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:21.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:18:21.578 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:21.578 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:21.592 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:21.599 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:21.602+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:21.664 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:21.666+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:21.705 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:21.706 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:21.724 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:21.724 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:21.744 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:21.744 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:21.752 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:21.752 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:21.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:21.768 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:21.770+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:21.768 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:21.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:21.791 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:21.791 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:21.808 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:21.808 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:21.871 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:21.874+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:21.897 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:21.897 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:21.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:21.897 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:21.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.897 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.897 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.675898109, 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:18:21.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:18:21.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.16349282600001, 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.909 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.163427923, 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:18:21.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:18:21.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.16328148400001, 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:21.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:18:21.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.16321236900001, 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.922 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:21.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:18:21.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.16260004399999, 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.928 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:18:21.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:18:21.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.16239563900001, 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:21.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:18:21.939 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 215.16235285299999, 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.940 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:18:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77139456500001, 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:18:21.951 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77130805499999, 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.952 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:21.957 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77123116600001, 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:18:21.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:18:21.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:21.963 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.771026326, 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:18:21.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:18:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77093804399999, 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.969 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:18:21.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77080721600001, 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.975 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:21.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:18:21.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77076156300001, 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.981 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:18:21.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:18:21.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77072313599999, 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:18:21.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.992 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.770593674, 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:18:21.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:21.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77057203199999, 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:18:22.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:18:22.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77055198799999, 2024-09-21T23:18:22.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:18:22.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:18:22.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:22.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 214.77046799600001, 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:18:22.015 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.016 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.016 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.016 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.016 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.016 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:22.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:22.018 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:22.016+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:22.118 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:22.116+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:22.194 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:22.278 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:22.278 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:22.278 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:22.280 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:22.280 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:22.276+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:22.381 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:22.382+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:22.448 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:22.448 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:22.448 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:22.448 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:22.448 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:22.462 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:22.481 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:22.482+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:22.581 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:22.581 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:22.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:22.581 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:22.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.581 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4713.0:597 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140428049451328 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 167.62402288199999, 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0177051100000001, 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4713", 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 597 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953806+0000", 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953808+0000", 2024-09-21T23:18:22.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.694e-06 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953817+0000", 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2730000000000008e-06 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953846+0000", 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8677e-05 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953864+0000", 2024-09-21T23:18:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7867e-05 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.953993+0000", 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012927199999999999 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954004+0000", 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0868e-05 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956749+0000", 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027450410000000001 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956785+0000", 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6062999999999997e-05 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968571+0000", 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0117856999999999 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.968771+0000", 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020026700000000001 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.971512+0000", 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027403879999999999 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4337.0:5316 3.1a 3:58748185:::rbd_header.10f1dc9013:head [watch reconnect cookie 140478314330672 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T23:18:22.587 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 167.62375589800001, 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0201274570000001, 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4337", 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 5316 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954073+0000", 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954074+0000", 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.108e-06 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954081+0000", 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0220000000000001e-06 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954092+0000", 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1265e-05 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954099+0000", 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4610000000000002e-06 2024-09-21T23:18:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954165+0000", 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5532999999999998e-05 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.954174+0000", 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2830000000000007e-06 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956880+0000", 2024-09-21T23:18:22.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0027055270000000001 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:34.956911+0000", 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1000000000000001e-05 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969453+0000", 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0125423769999999 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.969522+0000", 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8745000000000007e-05 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:15:35.974201+0000", 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046791360000000004 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26018 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7178240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.011427353, 2024-09-21T23:18:22.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086225320999999994, 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26018 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566402+0000", 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0999999999999999e-07 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566413+0000", 2024-09-21T23:18:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0451e-05 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566426+0000", 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2917e-05 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566434+0000", 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0269999999999995e-06 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652326+0000", 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085892242999999993 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652436+0000", 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109534 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652627+0000", 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019163900000000001 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.597 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26019 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7241728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.011384654, 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086368739, 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.598 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26019 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566445+0000", 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.82e-07 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566449+0000", 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4769999999999999e-06 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566462+0000", 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3495e-05 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566470+0000", 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6320000000000008e-06 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652667+0000", 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086197593000000003 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652722+0000", 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4796999999999997e-05 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652813+0000", 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1162999999999994e-05 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26020 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7194624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.011346034, 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086666531000000005, 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.602 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26020 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566483+0000", 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566484+0000", 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-07 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566487+0000", 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4979999999999998e-06 2024-09-21T23:18:22.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566498+0000", 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0137e-05 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566505+0000", 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3420000000000002e-06 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652854+0000", 2024-09-21T23:18:22.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086348840999999996 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.652960+0000", 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106122 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653150+0000", 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018988100000000001 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26021 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.01131416200001, 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086812151000000004, 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26021 2024-09-21T23:18:22.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566515+0000", 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566516+0000", 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4300000000000003e-07 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566519+0000", 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.411e-06 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566528+0000", 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4050000000000006e-06 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566537+0000", 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3839999999999997e-06 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653183+0000", 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086646551000000002 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653237+0000", 2024-09-21T23:18:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3810000000000001e-05 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653327+0000", 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0047e-05 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26022 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7211008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.011280938, 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087130956999999995, 2024-09-21T23:18:22.611 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26022 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566548+0000", 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566549+0000", 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4700000000000001e-07 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566552+0000", 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5590000000000001e-06 2024-09-21T23:18:22.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566565+0000", 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2384e-05 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566573+0000", 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0609999999999996e-06 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653368+0000", 2024-09-21T23:18:22.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086795106999999996 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653479+0000", 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110915 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653679+0000", 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020038400000000001 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26023 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7274496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.011245863, 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087292553999999994, 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.616 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26023 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566583+0000", 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566584+0000", 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.23e-07 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566588+0000", 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2909999999999999e-06 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566597+0000", 2024-09-21T23:18:22.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1919999999999994e-06 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566604+0000", 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0419999999999997e-06 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653700+0000", 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087096334999999997 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653785+0000", 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5185999999999994e-05 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653876+0000", 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0685000000000003e-05 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26024 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7227392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.011200905, 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087582778999999999, 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26024 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.566628+0000", 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567021+0000", 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00039275699999999998 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567036+0000", 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4944e-05 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567052+0000", 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5548000000000001e-05 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567062+0000", 2024-09-21T23:18:22.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.08e-05 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.653917+0000", 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086854953999999998 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654028+0000", 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110432 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654211+0000", 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000183344 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26025 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7290880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.010748838, 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087328506, 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26025 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567080+0000", 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567727+0000", 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064693599999999997 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567733+0000", 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0970000000000002e-06 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567750+0000", 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6928000000000001e-05 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567762+0000", 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1232e-05 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654257+0000", 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086495300999999997 2024-09-21T23:18:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654318+0000", 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1549000000000003e-05 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654409+0000", 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0463000000000004e-05 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26026 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7243776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.010034359, 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086953749999999996, 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26026 2024-09-21T23:18:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567795+0000", 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567796+0000", 2024-09-21T23:18:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567800+0000", 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4859999999999999e-06 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567814+0000", 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4121000000000001e-05 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567823+0000", 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9870000000000005e-06 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654448+0000", 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086624602999999994 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654556+0000", 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108334 2024-09-21T23:18:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654749+0000", 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192342 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26027 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7260160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.00999328899999, 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.087069503000000006, 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26027 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.567836+0000", 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568204+0000", 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000367897 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568210+0000", 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.438e-06 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568250+0000", 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9811999999999998e-05 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568261+0000", 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654764+0000", 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086502739999999995 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654816+0000", 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1473e-05 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654905+0000", 2024-09-21T23:18:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9790999999999995e-05 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26028 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7307264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.00953617099999, 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086945716000000006, 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26028 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568293+0000", 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568462+0000", 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016840599999999999 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568470+0000", 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3399999999999998e-06 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568484+0000", 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.382e-05 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568494+0000", 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0186999999999999e-05 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.654944+0000", 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086450250000000006 2024-09-21T23:18:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655050+0000", 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105538 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655239+0000", 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000189175 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26029 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7276544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.009299492, 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086893320999999996, 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26029 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568530+0000", 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568883+0000", 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00035306600000000002 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568889+0000", 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568903+0000", 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4749000000000001e-05 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568915+0000", 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1243999999999999e-05 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655274+0000", 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086359980000000003 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655328+0000", 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3801000000000002e-05 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655423+0000", 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4831000000000001e-05 2024-09-21T23:18:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26030 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7323648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.008883911, 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086820065000000002, 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26030 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.568945+0000", 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569555+0000", 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00060913500000000001 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569575+0000", 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0001e-05 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569590+0000", 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5285999999999999e-05 2024-09-21T23:18:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569604+0000", 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3934000000000001e-05 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655464+0000", 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085860669000000001 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655573+0000", 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108754 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655765+0000", 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019228600000000001 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26031 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7292928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.008206251, 2024-09-21T23:18:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086384313000000004, 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26031 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.569623+0000", 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570057+0000", 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00043403400000000003 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570064+0000", 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4860000000000002e-06 2024-09-21T23:18:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570082+0000", 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8740999999999999e-05 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570095+0000", 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2813000000000001e-05 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655812+0000", 2024-09-21T23:18:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085717160000000001 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.655884+0000", 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2082000000000005e-05 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656007+0000", 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012299699999999999 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26032 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 7309312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.00771496199999, 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234515999999997, 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26032 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570114+0000", 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570315+0000", 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000200189 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570320+0000", 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9739999999999999e-06 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570338+0000", 2024-09-21T23:18:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7711999999999999e-05 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570349+0000", 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0766e-05 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656052+0000", 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085702684000000001 2024-09-21T23:18:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656159+0000", 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000107785 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656349+0000", 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018940600000000001 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5157.0:26033 3.2 3:46f5f637:::rbd_data.10f1e9e481e5.0000000000000004:11 [read 6819840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 126.00745806400001, 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086234185000000005, 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5157", 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 26033 2024-09-21T23:18:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570371+0000", 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570783+0000", 2024-09-21T23:18:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041214099999999998 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570789+0000", 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1149999999999999e-06 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570805+0000", 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5421e-05 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.570816+0000", 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1199000000000001e-05 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656399+0000", 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.085583301000000001 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656485+0000", 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5865000000000005e-05 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:16:16.656605+0000", 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120143 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:816 3.19 e113/73 3:9a7728b2:::rbd_data.10f1215954cc.0000000000000001:head v 113'289, mlcod=111'287)", 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.298266964999996, 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.086446624999999999, 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279562+0000", 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.279564+0000", 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0899999999999999e-06 2024-09-21T23:18:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297328+0000", 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017763277000000001 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297375+0000", 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7645000000000002e-05 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297399+0000", 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3960000000000001e-05 2024-09-21T23:18:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297517+0000", 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000117649 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.297544+0000", 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7373999999999999e-05 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365887+0000", 2024-09-21T23:18:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068342904999999995 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366009+0000", 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000121725 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:820 3.14 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.295388263999996, 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089892505999999997, 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 820 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282441+0000", 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.282444+0000", 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7920000000000001e-06 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306724+0000", 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024280622000000002 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306764+0000", 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9808999999999998e-05 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306786+0000", 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1480000000000001e-05 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.306901+0000", 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000115642 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.307116+0000", 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021501699999999999 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 5", 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.309216+0000", 2024-09-21T23:18:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099779 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372256+0000", 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.063039922999999998 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372264+0000", 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2640000000000008e-06 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372299+0000", 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4631000000000001e-05 2024-09-21T23:18:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.372334+0000", 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4547e-05 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:22.676 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:22.677 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:22.678 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:22.678+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:22.779 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:22.782+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:22.803 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:22.803 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:22.803 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:22.804 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:22.817 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:22.879 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:22.876+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:22.884 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:18:22.884 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:18:22.885 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T23:18:22.885 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T23:18:22.885 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T23:18:22.885 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-21T23:18:22.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T23:18:22.957 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:22.957 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:22.957 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:22.957 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:22.957 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:22.974 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:23.017 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:23.016+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:23.021 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:23.021 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "" 2024-09-21T23:18:23.021 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:23.081 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:23.082+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:814 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.851798145000004, 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10676387800000001, 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:23.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 814 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251646+0000", 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.251647+0000", 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2440000000000001e-06 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276624+0000", 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024976743999999999 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276654+0000", 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9955e-05 2024-09-21T23:18:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276664+0000", 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0229e-05 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276744+0000", 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0589999999999999e-05 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276943+0000", 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019815800000000001 2024-09-21T23:18:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.278763+0000", 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0018208499999999999 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.346906+0000", 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.068142796000000005 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358283+0000", 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011376399000000001 2024-09-21T23:18:23.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358351+0000", 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8143000000000001e-05 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.358409+0000", 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8770000000000001e-05 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5823.0:815 3.1d 3:b9e165d3:::rbd_data.10f1215954cc.0000000000000002:head [write 0~0] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.826749186000001, 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089312631000000003, 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5823", 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/3160644173", 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 815 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276695+0000", 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276696+0000", 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276699+0000", 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3960000000000002e-06 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276704+0000", 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5630000000000002e-06 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.276707+0000", 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312035+0000", 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035328142 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312303+0000", 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026787600000000001 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.312815+0000", 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00051208999999999998 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:23.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365528+0000", 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0527124 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365942+0000", 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00041421600000000002 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.365977+0000", 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5553999999999999e-05 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.366007+0000", 2024-09-21T23:18:23.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9896e-05 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.5823.0:820 3.14 e113/25 3:2d45becd:::rbd_data.10f1215954cc.0000000000000003:head v 113'618, mlcod=113'616)", 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.793316016000006, 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060260949000000001, 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310128+0000", 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.310130+0000", 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7609999999999999e-06 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317321+0000", 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0071903150000000001 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317353+0000", 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1968000000000003e-05 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317367+0000", 2024-09-21T23:18:23.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3906e-05 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317487+0000", 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120444 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.317516+0000", 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.883e-05 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370322+0000", 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052805932 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:13.370389+0000", 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6792999999999996e-05 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1238 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.120 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.822565556000001, 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054982073999999999, 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1238 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280878+0000", 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280879+0000", 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8700000000000005e-07 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280883+0000", 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.912e-06 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280894+0000", 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1275e-05 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.280904+0000", 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.73e-06 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335407+0000", 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054502816000000003 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335499+0000", 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2589000000000004e-05 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.335860+0000", 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036096499999999998 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1341 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.819134048000002, 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055656854999999998, 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1341 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284310+0000", 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284311+0000", 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.06e-06 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284314+0000", 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.72e-06 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284328+0000", 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3380000000000001e-05 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284338+0000", 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0052e-05 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338191+0000", 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053852786999999999 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.338273+0000", 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2021000000000006e-05 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339967+0000", 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0016938350000000001 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.128 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1342 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 966656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.819094067999998, 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056087662000000003, 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1342 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.129 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284350+0000", 2024-09-21T23:18:23.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5400000000000001e-07 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284354+0000", 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.569e-06 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284364+0000", 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0710000000000001e-05 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284373+0000", 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7819999999999999e-06 2024-09-21T23:18:23.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340014+0000", 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055641057000000001 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340135+0000", 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120698 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340437+0000", 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00030229200000000001 2024-09-21T23:18:23.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1343 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 245760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.819056085, 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055970117, 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.133 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1343 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284388+0000", 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9200000000000001e-07 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284392+0000", 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8240000000000001e-06 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284404+0000", 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1584e-05 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284415+0000", 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1412e-05 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.339993+0000", 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055577519999999998 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340056+0000", 2024-09-21T23:18:23.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3127999999999996e-05 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340358+0000", 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000302057 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1344 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 393216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.819016720999997, 2024-09-21T23:18:23.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056410920000000003, 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1344 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284427+0000", 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284428+0000", 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6400000000000001e-07 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284432+0000", 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9060000000000004e-06 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284443+0000", 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.134e-05 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284452+0000", 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0340000000000002e-06 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340391+0000", 2024-09-21T23:18:23.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055938437000000001 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340464+0000", 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3744000000000004e-05 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340838+0000", 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000373595 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1345 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 671744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.141 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.818981121, 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056714902999999997, 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1345 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:23.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284463+0000", 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284464+0000", 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1540000000000001e-06 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284467+0000", 2024-09-21T23:18:23.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.63e-06 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284479+0000", 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.17e-05 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284488+0000", 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1870000000000003e-06 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340886+0000", 2024-09-21T23:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056398177000000001 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.340951+0000", 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4263000000000005e-05 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341178+0000", 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022679199999999999 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1346 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.818914648000003, 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057046119999999999, 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1346 2024-09-21T23:18:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284529+0000", 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284530+0000", 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0160000000000001e-06 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284535+0000", 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7960000000000003e-06 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284548+0000", 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3002e-05 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.284558+0000", 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8640000000000008e-06 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341212+0000", 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056654228000000001 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341281+0000", 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9061999999999997e-05 2024-09-21T23:18:23.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341575+0000", 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000294152 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1419 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 32768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.818067087999999, 2024-09-21T23:18:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056364545000000002, 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1419 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285377+0000", 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.06e-07 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285381+0000", 2024-09-21T23:18:23.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3079999999999999e-06 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285391+0000", 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.03e-05 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285400+0000", 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1959999999999997e-06 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341615+0000", 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056215380000000002 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341674+0000", 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8403000000000003e-05 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341741+0000", 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7352000000000002e-05 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1420 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.818031075999997, 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056531351000000001, 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1420 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.155 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285413+0000", 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3799999999999997e-07 2024-09-21T23:18:23.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285417+0000", 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0459999999999999e-06 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285429+0000", 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1402e-05 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285438+0000", 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2050000000000008e-06 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341791+0000", 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056353542999999999 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341861+0000", 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9338000000000006e-05 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341944+0000", 2024-09-21T23:18:23.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3178999999999998e-05 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1421 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.817991376999998, 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056621478000000003, 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1421 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285452+0000", 2024-09-21T23:18:23.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285453+0000", 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.066e-06 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285457+0000", 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285468+0000", 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0974e-05 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.285477+0000", 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0459999999999994e-06 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.341982+0000", 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056504285000000001 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342029+0000", 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7528999999999998e-05 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342074+0000", 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4647999999999997e-05 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1424 3.1c 3:3a52e0d0:::rbd_data.10f1abbced14.0000000000000018:28 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:23.163 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.815485252999999, 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054910207000000003, 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1424 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287958+0000", 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287959+0000", 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0139999999999999e-06 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287965+0000", 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.287989+0000", 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4511e-05 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288003+0000", 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3318e-05 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342685+0000", 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054682236000000002 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342764+0000", 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9122999999999994e-05 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.342869+0000", 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010467399999999999 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1448 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.814959051999999, 2024-09-21T23:18:23.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055210594000000002, 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1448 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288485+0000", 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288486+0000", 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4200000000000004e-07 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288490+0000", 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8979999999999998e-06 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288501+0000", 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1094000000000001e-05 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288509+0000", 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7499999999999992e-06 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343373+0000", 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05486365 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343441+0000", 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8119000000000003e-05 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343695+0000", 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025414099999999999 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1449 3.c 3:338a688e:::rbd_data.10f1abbced14.0000000000000001:28 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:23.171 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.814923020000002, 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055644921999999999, 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1449 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288521+0000", 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4600000000000005e-07 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288525+0000", 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9519999999999996e-06 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288536+0000", 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0862e-05 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288544+0000", 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.276e-06 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343752+0000", 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055208113000000003 2024-09-21T23:18:23.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343837+0000", 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4466000000000001e-05 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344166+0000", 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00032870700000000002 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1450 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 114688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.814887868, 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055497638000000002, 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1450 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288556+0000", 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288557+0000", 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288562+0000", 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7860000000000002e-06 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288574+0000", 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1637999999999999e-05 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288583+0000", 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8240000000000006e-06 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343733+0000", 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055150504000000003 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.343785+0000", 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1805999999999998e-05 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344054+0000", 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026842000000000001 2024-09-21T23:18:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1451 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.814849440000003, 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055892521000000001, 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1451 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288594+0000", 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288595+0000", 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0999999999999998e-07 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288599+0000", 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1999999999999996e-06 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288611+0000", 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1506e-05 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288619+0000", 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7790000000000001e-06 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344089+0000", 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055469472999999998 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344170+0000", 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1287999999999998e-05 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344487+0000", 2024-09-21T23:18:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031666499999999999 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1452 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.814813258999997, 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056290277999999999, 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1452 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288630+0000", 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288631+0000", 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288635+0000", 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0339999999999998e-06 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288646+0000", 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1107999999999999e-05 2024-09-21T23:18:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288656+0000", 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3319999999999998e-06 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344531+0000", 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055875073999999997 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344606+0000", 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4838000000000004e-05 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344921+0000", 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031523599999999999 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.5768.0:1453 3.1c 3:3ccdff17:::rbd_data.10f1abbced14.0000000000000002:28 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.814777403000001, 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056627027000000003, 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.5768", 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.170:0/575929480", 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1453 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288666+0000", 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288667+0000", 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2799999999999996e-07 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.288671+0000", 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.293991+0000", 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053204910000000001 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.294012+0000", 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1019999999999999e-05 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.344962+0000", 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050949529 2024-09-21T23:18:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345048+0000", 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6263999999999999e-05 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:17:41.345293+0000", 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000245135 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:23.193 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:23.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:23.194 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:23.194+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:23.294 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:23.294+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:23.321 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:23.321 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:23.321 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:23.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:23.396 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:23.392+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:23.496 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:23.498+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:23.503 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:23.503 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:23.503 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:23.503 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:23.503 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:23.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:23.598 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:23.596+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:23.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 141.52909206999999, 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:23.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:18:23.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:18:23.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:18:23.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.46383731200001, 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.06649518, 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.06312749099999, 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:18:23.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:18:23.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.05901791100001, 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:18:23.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.05522499, 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99130911200001, 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:18:23.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99128231099999, 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:23.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99125614600001, 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:18:23.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99122769600001, 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.729 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:18:23.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99113698799999, 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:18:23.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:18:23.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.736 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.991105409, 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:18:23.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:18:23.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.740 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.991067719, 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:18:23.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:18:23.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99103732899999, 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:18:23.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:18:23.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99099391199999, 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:23.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.99095928700001, 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.98770578700001, 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.987671205, 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:18:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.771926806, 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:23.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:18:23.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:23.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:23.771 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 47.16295161, 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:23.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:18:23.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:18:23.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:23.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:23.777 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:23.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:23.779 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:23.776+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:23.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:18:23.879 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:23.882+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:23.892 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-21T23:18:23.918 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:23.918 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:23.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:23.983 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:23.986+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:23.987 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:23.987 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:24.006 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:24.006 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:24.023 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:24.045 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:24.046 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:24.067 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:24.067 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:24.080 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:24.082+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:24.091 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:24.091 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:24.104 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:24.104 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:24.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:24.207 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:24.210+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:24.282 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:24.280+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:24.284 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:24.285 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:24.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:24.387 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:24.390+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:24.468 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:24.468 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:24.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:24.527 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:24.530+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:24.585 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:24.580+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:24.655 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:24.655 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:24.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:24.685 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:24.680+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:24.786 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:24.786+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:24.796 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:24.796 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:24.811 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:24.887 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:24.884+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:24.952 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:24.952 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:24.967 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:24.988 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:24.990+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:25.089 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:25.084+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:25.134 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:25.134 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:25.150 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:25.194 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:25.192+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:25.291 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:25.294+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:25.320 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:25.320 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:25.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:25.392 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:25.388+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:25.457 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:25.458 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:25.473 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:25.493 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:25.494+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:25.594 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:25.594+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:25.638 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:25.639 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:25.653 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:25.695 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:25.698+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:25.772 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:25.772 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:25.788 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:25.796 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:25.798+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:25.897 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:25.892+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:25.911 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:25.911 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:25.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T23:18:25.930 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:26.017 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:26.016+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:26.099 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:26.096+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:26.103 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:26.104 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:26.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T23:18:26.119 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:26.205 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:26.204+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:26.250 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:26.250 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:26.268 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:26.285 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:26.285 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:26.285 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:26.285 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:26.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:26.304 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:26.304 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:26.324 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:26.324 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:26.342 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:26.340+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:26.344 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:26.344 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:26.407 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:26.410+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:26.419 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:26.419 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:26.419 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:26.419 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:26.433 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:26.519 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:26.522+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:26.592 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:26.593 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:26.593 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:26.593 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:26.593 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:26.604 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:26.600+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:26.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:26.704 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:26.700+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:26.764 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4900.0:8098 3.15 3.655007d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 144.627348797, 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0088484950000001, 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4900", 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2636938889", 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8098 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161384+0000", 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161386+0000", 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2330000000000001e-06 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161394+0000", 2024-09-21T23:18:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6290000000000001e-06 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161408+0000", 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3818999999999999e-05 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161419+0000", 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161501+0000", 2024-09-21T23:18:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1491999999999998e-05 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.161508+0000", 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059999999999999e-06 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175137+0000", 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013628415 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:02.175155+0000", 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7935e-05 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170216+0000", 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99506123300000004 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:03.170233+0000", 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6801000000000001e-05 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4337.0:9573 3.b 3:d690c289:::rbd_header.10f177c8c3f9:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.56209403899999, 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99417704399999995, 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4337", 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/436626266", 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9573 2024-09-21T23:18:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226639+0000", 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226642+0000", 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2430000000000001e-06 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226657+0000", 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4976e-05 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226704+0000", 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6807000000000001e-05 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226718+0000", 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4528999999999999e-05 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226813+0000", 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4326000000000005e-05 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:06.226824+0000", 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1003e-05 2024-09-21T23:18:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220436+0000", 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99361226800000002 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220568+0000", 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000132428 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:07.220816+0000", 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024746399999999997 2024-09-21T23:18:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1167 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 135.16475190700001, 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083103342999999996, 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1167 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.623981+0000", 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626759+0000", 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027780510000000001 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626767+0000", 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.498e-06 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626787+0000", 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0051000000000002e-05 2024-09-21T23:18:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.626799+0000", 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2277e-05 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706848+0000", 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.08004936 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.706958+0000", 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010976600000000001 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.707084+0000", 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012634000000000001 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1178 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 1036288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 135.16138421799999, 2024-09-21T23:18:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083266026000000007, 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1178 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.627349+0000", 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630867+0000", 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003517924 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630877+0000", 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9799999999999993e-06 2024-09-21T23:18:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630903+0000", 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.630914+0000", 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0275000000000001e-05 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710333+0000", 2024-09-21T23:18:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079419208000000005 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710465+0000", 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013262199999999999 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.710615+0000", 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000149302 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1191 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 135.15727463799999, 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083294858999999999, 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1191 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.631458+0000", 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634733+0000", 2024-09-21T23:18:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003274674 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634740+0000", 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-06 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634755+0000", 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.505e-05 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.634764+0000", 2024-09-21T23:18:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2599999999999994e-06 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714492+0000", 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079728212000000007 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714609+0000", 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117025 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.714753+0000", 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014370799999999999 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:1203 3.17 3:ee27287e:::rbd_data.10f1e9e481e5.0000000000000010:11 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 135.15348171700001, 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083320367000000006, 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1203 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.635251+0000", 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638847+0000", 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003596274 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638856+0000", 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9400000000000008e-06 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638882+0000", 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5332000000000001e-05 2024-09-21T23:18:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.638897+0000", 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5375000000000001e-05 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718325+0000", 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.079427672000000005 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718442+0000", 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011729499999999999 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:11.718572+0000", 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129479 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8011 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4124672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.08956583899999, 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083538555, 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8011 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699167+0000", 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699168+0000", 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0299999999999999e-07 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699170+0000", 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699179+0000", 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699185+0000", 2024-09-21T23:18:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4309999999999999e-06 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782332+0000", 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083147083999999996 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782495+0000", 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016257199999999999 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782706+0000", 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000210702 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8012 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4093952~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.089539038, 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083775094999999994, 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8012 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699194+0000", 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-07 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699197+0000", 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9409999999999999e-06 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699206+0000", 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2490000000000001e-06 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699212+0000", 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3430000000000001e-06 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782787+0000", 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083575151 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782883+0000", 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5650999999999996e-05 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.782969+0000", 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6258e-05 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8013 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4141056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.08951287299999, 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084173640999999993, 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8013 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699220+0000", 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699221+0000", 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9999999999999995e-07 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699224+0000", 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7599999999999998e-06 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699234+0000", 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8220000000000002e-06 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699240+0000", 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2910000000000004e-06 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783038+0000", 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083798151000000001 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783198+0000", 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000159925 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783394+0000", 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000195692 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8014 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4157440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.08948442299999, 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084713574999999999, 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8014 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699249+0000", 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699270+0000", 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1209999999999999e-05 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699289+0000", 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8433000000000001e-05 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699297+0000", 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6379999999999993e-06 2024-09-21T23:18:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783464+0000", 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084166830999999998 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783715+0000", 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025072799999999998 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.783962+0000", 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000247252 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8015 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4110336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.089393715, 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085005038000000005, 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8015 2024-09-21T23:18:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699339+0000", 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699340+0000", 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0800000000000004e-07 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699344+0000", 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8920000000000002e-06 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699353+0000", 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6999999999999997e-06 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699360+0000", 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3900000000000004e-06 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784003+0000", 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642889999999998 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784159+0000", 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015636900000000001 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784344+0000", 2024-09-21T23:18:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018498899999999999 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8016 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4173824~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.08936213600001, 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085480531999999998, 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8016 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699371+0000", 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699372+0000", 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699375+0000", 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0970000000000002e-06 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699391+0000", 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6447999999999999e-05 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699398+0000", 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2949999999999996e-06 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784444+0000", 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085045536000000005 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784610+0000", 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016578900000000001 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784851+0000", 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024163099999999999 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8017 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4126720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.08932444600001, 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085949261999999998, 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8017 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699408+0000", 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699409+0000", 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4000000000000001e-07 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699413+0000", 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.394e-06 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699421+0000", 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0390000000000005e-06 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699429+0000", 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.619e-06 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.784936+0000", 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085507147000000006 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785130+0000", 2024-09-21T23:18:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000193582 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785358+0000", 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000227741 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8018 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4190208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.089294056, 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086450452999999997, 2024-09-21T23:18:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8018 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699439+0000", 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7700000000000005e-07 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699443+0000", 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2949999999999998e-06 2024-09-21T23:18:26.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699462+0000", 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9602e-05 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699470+0000", 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8839999999999994e-06 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785491+0000", 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086021200000000006 2024-09-21T23:18:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785661+0000", 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169323 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785889+0000", 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000228672 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8019 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4143104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.089250639, 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086906516000000003, 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8019 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699482+0000", 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699483+0000", 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6100000000000001e-07 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699487+0000", 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9829999999999998e-06 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699495+0000", 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699507+0000", 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1761e-05 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.785984+0000", 2024-09-21T23:18:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.086476490000000003 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786148+0000", 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016472599999999999 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786389+0000", 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024046500000000001 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8020 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4159488~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.08921601399999, 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.087385070999999995, 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8020 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.699517+0000", 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702696+0000", 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031787270000000001 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702701+0000", 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0259999999999998e-06 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702713+0000", 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2245999999999999e-05 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702722+0000", 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6510000000000007e-06 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786473+0000", 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.083751333999999997 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786642+0000", 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169352 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.786902+0000", 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025973499999999998 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8021 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4175872~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.08596251399999, 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084642207999999997, 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8021 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702770+0000", 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702772+0000", 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1620000000000001e-06 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702775+0000", 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.777e-06 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702786+0000", 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0757000000000001e-05 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702793+0000", 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2930000000000003e-06 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787023+0000", 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084229264999999998 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787204+0000", 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018106900000000001 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787413+0000", 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020888499999999999 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:8022 3.6 3:665446b7:::rbd_data.10f1e9e481e5.0000000000000012:11 [read 4192256~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.085927932, 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.085058889999999998, 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8022 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702805+0000", 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702806+0000", 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.51e-07 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702809+0000", 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3459999999999998e-06 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702817+0000", 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4079999999999998e-06 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.702824+0000", 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4840000000000001e-06 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787506+0000", 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.084682296000000004 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787663+0000", 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000156436 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:12.787864+0000", 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020136899999999999 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:29560 3.12 3:4ce14d62:::rbd_data.10f13fc20328.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.87018353299999, 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85992542800000005, 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 29560 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918549+0000", 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918551+0000", 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.274e-06 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918557+0000", 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.754e-06 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918575+0000", 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7354e-05 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918588+0000", 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989999999999999e-05 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918686+0000", 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8054999999999995e-05 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.918694+0000", 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-21T23:18:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769047+0000", 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.85035304899999997 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769177+0000", 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013006199999999999 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.769343+0000", 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016625200000000001 2024-09-21T23:18:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.771146+0000", 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001803245 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778341+0000", 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007194659 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778420+0000", 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8770999999999995e-05 2024-09-21T23:18:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:18.778475+0000", 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5044999999999999e-05 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6150.0:607 3.b 3:d6d61d6a:::rbd_data.10f1abbced14.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 671744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.261208336999999, 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.650225787, 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6150", 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/1141027037", 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527525+0000", 2024-09-21T23:18:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527527+0000", 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1040000000000001e-06 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527538+0000", 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.092e-05 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527569+0000", 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1516000000000001e-05 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527584+0000", 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527708+0000", 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012408499999999999 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:36.527726+0000", 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8012999999999998e-05 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.173877+0000", 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.64615039399999996 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174106+0000", 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022880099999999999 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-21T23:18:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.174425+0000", 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031894600000000002 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.176312+0000", 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018873169999999999 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177599+0000", 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001286726 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177689+0000", 2024-09-21T23:18:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0116000000000003e-05 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:17:37.177750+0000", 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1668999999999995e-05 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:26.878 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:26.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:26.880 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:26.876+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:26.980 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:26.982+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:27.017 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:27.017 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:27.018 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:27.018 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:27.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:27.082 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:27.080+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:27.182 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:27.182+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:27.194 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:27.195 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:27.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:27.283 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:27.280+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27831 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60834076099999, 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044814658, 2024-09-21T23:18:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27831 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761626+0000", 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9299999999999998e-07 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761629+0000", 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761637+0000", 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9039999999999992e-06 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761643+0000", 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3300000000000004e-06 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762448+0000", 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080524700000000004 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762454+0000", 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0680000000000002e-06 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806431+0000", 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043976443 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806440+0000", 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.499e-06 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27832 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60829550400001, 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448218380000001, 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27832 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761671+0000", 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761672+0000", 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.02e-07 2024-09-21T23:18:27.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761676+0000", 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5920000000000002e-06 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761690+0000", 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3532e-05 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761699+0000", 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7339999999999997e-06 2024-09-21T23:18:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762483+0000", 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078419199999999996 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762489+0000", 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806484+0000", 2024-09-21T23:18:27.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043995445 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806493+0000", 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3520000000000007e-06 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27833 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:27.377 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60824588200001, 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044811529, 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27833 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:27.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761721+0000", 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2300000000000001e-07 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761724+0000", 2024-09-21T23:18:27.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761734+0000", 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1949999999999992e-06 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761741+0000", 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762517+0000", 2024-09-21T23:18:27.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077634299999999998 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762523+0000", 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8379999999999998e-06 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806523+0000", 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440003259999999 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806532+0000", 2024-09-21T23:18:27.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27834 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60821618700001, 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04483379, 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27834 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761750+0000", 2024-09-21T23:18:27.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761751+0000", 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.65e-07 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761753+0000", 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.503e-06 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761761+0000", 2024-09-21T23:18:27.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0649999999999999e-06 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761767+0000", 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5899999999999998e-06 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762555+0000", 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078788800000000004 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762561+0000", 2024-09-21T23:18:27.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4980000000000003e-06 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806573+0000", 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044011349 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806584+0000", 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1532000000000001e-05 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27835 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60818325899999, 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044841176, 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27835 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761783+0000", 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761784+0000", 2024-09-21T23:18:27.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761793+0000", 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761802+0000", 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2590000000000006e-06 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761808+0000", 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6919999999999999e-06 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762591+0000", 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078307799999999999 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762597+0000", 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1260000000000003e-06 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806616+0000", 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018482 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806624+0000", 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7129999999999994e-06 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27836 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:27.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60814896299999, 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448391619999999, 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27836 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761817+0000", 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761818+0000", 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9200000000000002e-07 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761820+0000", 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.481e-06 2024-09-21T23:18:27.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761827+0000", 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761834+0000", 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3409999999999999e-06 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762624+0000", 2024-09-21T23:18:27.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079024300000000004 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762630+0000", 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1750000000000002e-06 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806648+0000", 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044018138 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806657+0000", 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27837 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.608124457, 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044914761, 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27837 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761842+0000", 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2300000000000002e-07 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761845+0000", 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.413e-06 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761852+0000", 2024-09-21T23:18:27.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7739999999999999e-06 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761857+0000", 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4909999999999996e-06 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762661+0000", 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000804184 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762669+0000", 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806748+0000", 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04407881 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806757+0000", 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0429999999999996e-06 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27838 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.608096467, 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04493642, 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.401 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27838 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761870+0000", 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761873+0000", 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.565e-06 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761880+0000", 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1110000000000002e-06 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761885+0000", 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2599999999999996e-06 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762707+0000", 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082208699999999999 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762714+0000", 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8870000000000003e-06 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806796+0000", 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440821440000001 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806806+0000", 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9480000000000003e-06 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27839 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.405 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60807343900001, 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449556170000001, 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27839 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761893+0000", 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761896+0000", 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2740000000000002e-06 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761902+0000", 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6819999999999997e-06 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761907+0000", 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9749999999999998e-06 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762742+0000", 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083447000000000003 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762748+0000", 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9340000000000003e-06 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806839+0000", 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044091737 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806849+0000", 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27840 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.608051929, 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449499769999999, 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27840 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761915+0000", 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5499999999999999e-07 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761917+0000", 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5299999999999999e-06 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761924+0000", 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.613e-06 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761929+0000", 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9110000000000001e-06 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762772+0000", 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084354099999999995 2024-09-21T23:18:27.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762778+0000", 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6629999999999998e-06 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806860+0000", 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440819509999999 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806865+0000", 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4129999999999999e-06 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27841 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60801638800001, 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044939029, 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27841 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761950+0000", 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761951+0000", 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761953+0000", 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761961+0000", 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8660000000000006e-06 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761967+0000", 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0430000000000004e-06 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762805+0000", 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083733400000000002 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762811+0000", 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5520000000000001e-06 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806881+0000", 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044069506 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806889+0000", 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3340000000000002e-06 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27842 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60798875099999, 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449249549999999, 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27842 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761978+0000", 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000002e-07 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761981+0000", 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8550000000000002e-06 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761988+0000", 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0539999999999997e-06 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.761993+0000", 2024-09-21T23:18:27.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4890000000000003e-06 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762838+0000", 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084414699999999995 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762844+0000", 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8799999999999996e-06 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806899+0000", 2024-09-21T23:18:27.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044055019 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806903+0000", 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1450000000000001e-06 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27843 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.607953587, 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0449130639999999, 2024-09-21T23:18:27.424 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27843 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762013+0000", 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.58e-07 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762016+0000", 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9340000000000002e-06 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762024+0000", 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6520000000000006e-06 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762030+0000", 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9059999999999999e-06 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762869+0000", 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083923600000000004 2024-09-21T23:18:27.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762875+0000", 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6219999999999997e-06 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806918+0000", 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440429309999999 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806926+0000", 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3250000000000008e-06 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27844 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60791855599999, 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448916340000001, 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27844 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762048+0000", 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7300000000000001e-07 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762051+0000", 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762059+0000", 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0420000000000003e-06 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762065+0000", 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7060000000000001e-06 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762903+0000", 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083853599999999997 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762909+0000", 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9549999999999997e-06 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806935+0000", 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440259030000001 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806940+0000", 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2180000000000001e-06 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27845 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.607883171, 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448915000000001, 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.434 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27845 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762083+0000", 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762084+0000", 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762086+0000", 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7530000000000002e-06 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762094+0000", 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7449999999999995e-06 2024-09-21T23:18:27.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762100+0000", 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762936+0000", 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083576600000000005 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762941+0000", 2024-09-21T23:18:27.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7919999999999998e-06 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806965+0000", 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04402361 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806975+0000", 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8449999999999998e-06 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27846 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60784298900001, 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044864115, 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27846 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762123+0000", 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762124+0000", 2024-09-21T23:18:27.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762127+0000", 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7149999999999998e-06 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762134+0000", 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6629999999999993e-06 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762140+0000", 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762970+0000", 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083059799999999999 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762976+0000", 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-21T23:18:27.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806983+0000", 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0440068870000001 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.806988+0000", 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4660000000000001e-06 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27847 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.607805861, 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448626489999999, 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27847 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762161+0000", 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762172+0000", 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0487e-05 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762180+0000", 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.14e-06 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762190+0000", 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0062e-05 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763040+0000", 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000850499 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763048+0000", 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5310000000000004e-06 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807014+0000", 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439666059999999 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807023+0000", 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8980000000000002e-06 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27848 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60776578599999, 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448359920000001, 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27848 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762201+0000", 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3799999999999998e-07 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762204+0000", 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762212+0000", 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3310000000000004e-06 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762218+0000", 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.592e-06 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763083+0000", 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086529399999999996 2024-09-21T23:18:27.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763089+0000", 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807033+0000", 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043943555 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807037+0000", 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0960000000000003e-06 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27849 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.607741242, 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0448466080000001, 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.453 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27849 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762225+0000", 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762226+0000", 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2899999999999999e-07 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762228+0000", 2024-09-21T23:18:27.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1849999999999998e-06 2024-09-21T23:18:27.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762234+0000", 2024-09-21T23:18:27.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-21T23:18:27.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762239+0000", 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3469999999999998e-06 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763114+0000", 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087495599999999998 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763121+0000", 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4810000000000003e-06 2024-09-21T23:18:27.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807063+0000", 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0439421090000001 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807072+0000", 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7909999999999993e-06 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5157.0:27850 3.2 3.ec6faf62 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 130.60770345700001, 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.044823069, 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5157", 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/2189472469", 2024-09-21T23:18:27.460 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 27850 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:27.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762263+0000", 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762266+0000", 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5289999999999998e-06 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762273+0000", 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2320000000000004e-06 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.762280+0000", 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7270000000000001e-06 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763146+0000", 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086600700000000002 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:16.763151+0000", 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.383e-06 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807081+0000", 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.043929924 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:16:17.807086+0000", 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8559999999999998e-06 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:27.464 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:27.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:27.466 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:27.466+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:27.566 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:27.566+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:27.634 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:27.634 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:27.635 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-21T23:18:27.635 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:27.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:27.668 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:27.664+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:27.768 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:27.770+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:27.810 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:27.810 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-21T23:18:27.810 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-21T23:18:27.810 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-21T23:18:27.811 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:27.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:27.871 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:27.874+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:27.970 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:27.970+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:28.010 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:28.010 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-21T23:18:28.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:550 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4063232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08268304500001, 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063627068, 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 550 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927774+0000", 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927777+0000", 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0259999999999999e-06 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927787+0000", 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5740000000000002e-06 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.927794+0000", 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2230000000000001e-06 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975590+0000", 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047795239000000003 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975617+0000", 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7093e-05 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991156+0000", 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.015539703 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991264+0000", 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010768900000000001 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.991401+0000", 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136861 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:553 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 540672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.081662198, 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0634989530000001, 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 553 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928795+0000", 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928800+0000", 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.977e-06 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928812+0000", 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.155e-05 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928842+0000", 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9675999999999998e-05 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975965+0000", 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047123024999999999 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.975974+0000", 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1020000000000003e-06 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992050+0000", 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0160758409999999 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992154+0000", 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104814 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992294+0000", 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139145 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:554 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08160252499999, 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.063736091, 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.021 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 554 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928854+0000", 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928855+0000", 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928859+0000", 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999996e-06 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928898+0000", 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8883e-05 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928929+0000", 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1269999999999997e-05 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976068+0000", 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047139028999999999 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976086+0000", 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8187000000000001e-05 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992348+0000", 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0162615690000001 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992457+0000", 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108978 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992590+0000", 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133485 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:555 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 933888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.081512549, 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0639371470000001, 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.026 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 555 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928944+0000", 2024-09-21T23:18:28.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928945+0000", 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3399999999999998e-07 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928950+0000", 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7530000000000001e-06 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928974+0000", 2024-09-21T23:18:28.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4170000000000001e-05 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928983+0000", 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5529999999999993e-06 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976159+0000", 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047176728000000001 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976179+0000", 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9505e-05 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992642+0000", 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.016462733 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992745+0000", 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010322100000000001 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992881+0000", 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013664999999999999 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:556 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1130496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08146263899999, 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064183189, 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 556 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928994+0000", 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.928995+0000", 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5600000000000005e-07 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929007+0000", 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2267e-05 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929018+0000", 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1107000000000001e-05 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929039+0000", 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0630000000000001e-05 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976272+0000", 2024-09-21T23:18:28.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047232784999999999 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976293+0000", 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1158999999999999e-05 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.992934+0000", 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0166414559999999 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993045+0000", 2024-09-21T23:18:28.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110256 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993177+0000", 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013277300000000001 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:557 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1327104~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.081406499, 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0644155550000001, 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 557 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929050+0000", 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929051+0000", 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5300000000000003e-07 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929064+0000", 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3037000000000001e-05 2024-09-21T23:18:28.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929075+0000", 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929083+0000", 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7840000000000004e-06 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976356+0000", 2024-09-21T23:18:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047273612 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976364+0000", 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5549999999999997e-06 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993231+0000", 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168667870000001 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993331+0000", 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100191 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993466+0000", 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013491999999999999 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:558 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.081352716, 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0646555790000001, 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 558 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929104+0000", 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929105+0000", 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9299999999999997e-07 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929108+0000", 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6200000000000001e-06 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929119+0000", 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0509999999999999e-05 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929127+0000", 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6499999999999996e-06 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976450+0000", 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047323581000000003 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976471+0000", 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0916000000000001e-05 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993519+0000", 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0170475830000001 2024-09-21T23:18:28.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993625+0000", 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106613 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993760+0000", 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000134413 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.046 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:559 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08132044000001, 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0649058689999999, 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 559 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929136+0000", 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929137+0000", 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-07 2024-09-21T23:18:28.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929140+0000", 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.207e-06 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929150+0000", 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3489999999999998e-06 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929157+0000", 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7400000000000004e-06 2024-09-21T23:18:28.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976534+0000", 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047376232999999997 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976540+0000", 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5830000000000003e-06 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993813+0000", 2024-09-21T23:18:28.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0172724879999999 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.993909+0000", 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5964999999999993e-05 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994042+0000", 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013364400000000001 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:560 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08129016000001, 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0651690469999999, 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 560 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.052 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929167+0000", 2024-09-21T23:18:28.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929171+0000", 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1870000000000001e-06 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929180+0000", 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6600000000000007e-06 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929187+0000", 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8530000000000003e-06 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976616+0000", 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047429374000000003 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976638+0000", 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2079999999999999e-05 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994094+0000", 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.017455601 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994201+0000", 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106848 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994336+0000", 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013479799999999999 2024-09-21T23:18:28.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:561 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.081260504, 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0654160690000001, 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 561 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929196+0000", 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929197+0000", 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929200+0000", 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929210+0000", 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6660000000000003e-06 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929217+0000", 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2640000000000002e-06 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976704+0000", 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047487276000000002 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976711+0000", 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7850000000000003e-06 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994386+0000", 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0176752069999999 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994483+0000", 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6787999999999996e-05 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994612+0000", 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012932899999999999 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.061 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:562 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2310144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08123053700001, 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0656801389999999, 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 562 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929226+0000", 2024-09-21T23:18:28.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929227+0000", 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6300000000000005e-07 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929232+0000", 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5639999999999998e-06 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929241+0000", 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7899999999999994e-06 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929249+0000", 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3159999999999999e-06 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976786+0000", 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047537072999999999 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976803+0000", 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7447000000000001e-05 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994666+0000", 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0178632750000001 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994773+0000", 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010642899999999999 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994906+0000", 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013358200000000001 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:563 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08115270799999, 2024-09-21T23:18:28.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0659123770000001, 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 563 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929304+0000", 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929305+0000", 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1699999999999997e-07 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929309+0000", 2024-09-21T23:18:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6899999999999998e-06 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929332+0000", 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3105999999999999e-05 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929340+0000", 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976866+0000", 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047525755000000003 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976873+0000", 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9029999999999998e-06 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.994962+0000", 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180896829999999 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995074+0000", 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011177500000000001 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995217+0000", 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014232099999999999 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:564 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.081105001, 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0661581170000001, 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 564 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929352+0000", 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929353+0000", 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929356+0000", 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3569999999999998e-06 2024-09-21T23:18:28.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929366+0000", 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6930000000000002e-06 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929373+0000", 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1400000000000002e-06 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976906+0000", 2024-09-21T23:18:28.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047533543999999997 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976913+0000", 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3779999999999998e-06 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995268+0000", 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018355779 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995377+0000", 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108303 2024-09-21T23:18:28.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995510+0000", 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013325699999999999 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:565 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08107480499999, 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066362684, 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 565 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929382+0000", 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929383+0000", 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5700000000000001e-07 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929386+0000", 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.033e-06 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929395+0000", 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3000000000000007e-06 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929403+0000", 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.976995+0000", 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047592409000000002 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977015+0000", 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.031e-05 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995564+0000", 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0185490070000001 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995661+0000", 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6711000000000003e-05 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995745+0000", 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3560000000000006e-05 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:566 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08104488800001, 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06649672, 2024-09-21T23:18:28.082 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 566 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929412+0000", 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929413+0000", 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929416+0000", 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.14e-06 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929425+0000", 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5000000000000005e-06 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929432+0000", 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977077+0000", 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047644830999999999 2024-09-21T23:18:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977084+0000", 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9580000000000001e-06 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995772+0000", 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0186879040000001 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995822+0000", 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9793000000000003e-05 2024-09-21T23:18:28.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995909+0000", 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6771e-05 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:567 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08101508799999, 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0667860600000001, 2024-09-21T23:18:28.087 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 567 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929442+0000", 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929445+0000", 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929455+0000", 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5270000000000004e-06 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929462+0000", 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1389999999999998e-06 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977166+0000", 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047703365999999997 2024-09-21T23:18:28.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977186+0000", 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0339999999999998e-05 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.995969+0000", 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.018783156 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996087+0000", 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118424 2024-09-21T23:18:28.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996228+0000", 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014036499999999999 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:568 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08098515500001, 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669787040000001, 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.092 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 568 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929472+0000", 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5000000000000002e-07 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929502+0000", 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9298999999999998e-05 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929516+0000", 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4504e-05 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929525+0000", 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4360000000000002e-06 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977250+0000", 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047725621000000003 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977263+0000", 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3215e-05 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996288+0000", 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190249200000001 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996353+0000", 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4639000000000002e-05 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996450+0000", 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7419999999999999e-05 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:569 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08092019200001, 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.06720601, 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 569 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929537+0000", 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6000000000000003e-07 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929541+0000", 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5899999999999999e-06 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929551+0000", 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.713e-06 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929558+0000", 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0530000000000001e-06 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977362+0000", 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047804275 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977382+0000", 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0417000000000001e-05 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996500+0000", 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0191172399999999 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996606+0000", 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105812 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996743+0000", 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013715 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.101 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:570 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 3883008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.080889716, 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.067365452, 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 570 2024-09-21T23:18:28.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929567+0000", 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929568+0000", 2024-09-21T23:18:28.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-07 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929571+0000", 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1499999999999999e-06 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929580+0000", 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2229999999999996e-06 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929587+0000", 2024-09-21T23:18:28.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1029999999999996e-06 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977451+0000", 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047863294000000001 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977459+0000", 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0080000000000002e-06 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996785+0000", 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0193267619999999 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996844+0000", 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8656999999999998e-05 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996933+0000", 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8601000000000001e-05 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4713.0:571 3.7 3:e7941950:::rbd_data.10f1163c318d.0000000000000000:8 [read 4079616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 173.08086055199999, 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0676297319999999, 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4713", 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.170:0/620841778", 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 571 2024-09-21T23:18:28.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929596+0000", 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929597+0000", 2024-09-21T23:18:28.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4600000000000004e-07 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929600+0000", 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1439999999999999e-06 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929610+0000", 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6560000000000004e-06 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.929617+0000", 2024-09-21T23:18:28.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2130000000000002e-06 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977544+0000", 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.047927354999999998 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:34.977567+0000", 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-21T23:18:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.996980+0000", 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.019413256 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997089+0000", 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010865399999999999 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-21T23:15:35.997226+0000", 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137095 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T23:18:28.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T23:18:28.112 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:28.112 INFO:tasks.thrashosds.thrasher:in_osds: [0, 4, 5, 1] out_osds: [3, 2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 0] 2024-09-21T23:18:28.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:18:28.113 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:28.114 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:28.112+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:28.214 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:28.214+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:28.246 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:28.246 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:28.246 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-21T23:18:28.246 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:28.260 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:28.314 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:28.314+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:28.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T23:18:28.382 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:28.382 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-21T23:18:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-21T23:18:28.383 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-21T23:18:28.383 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:28.398 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:28.445 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:28.444+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:28.516 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:28.518+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:28.519 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-21T23:18:28.519 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T23:18:28.523 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-21T23:18:28.523 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-21T23:18:28.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 222.30279165100001, 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.185589378, 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.524 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 1 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218410+0000", 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218412+0000", 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1739999999999999e-06 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218425+0000", 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.329e-05 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218448+0000", 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3054999999999999e-05 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218474+0000", 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5242999999999999e-05 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218615+0000", 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014083999999999999 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for peered", 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.218631+0000", 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6517000000000001e-05 2024-09-21T23:18:28.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.400954+0000", 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.18232274000000001 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401572+0000", 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061833799999999996 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.401961+0000", 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000388957 2024-09-21T23:18:28.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403204+0000", 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001243353 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403797+0000", 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059268600000000004 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403935+0000", 2024-09-21T23:18:28.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000137823 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.403999+0000", 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4362000000000005e-05 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:28.530 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.79038636799999, 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014001356, 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 111 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:28.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730815+0000", 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730816+0000", 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8400000000000003e-07 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730828+0000", 2024-09-21T23:18:28.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1749999999999999e-05 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730849+0000", 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1576000000000001e-05 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730858+0000", 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3760000000000008e-06 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740522+0000", 2024-09-21T23:18:28.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096644480000000008 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740692+0000", 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000169739 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.740897+0000", 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000204718 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744661+0000", 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037640310000000001 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744681+0000", 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.031e-05 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744754+0000", 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2655000000000001e-05 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744817+0000", 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2969e-05 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.79032146500001, 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015301692, 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.536 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 112 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730880+0000", 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730881+0000", 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4600000000000003e-07 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730977+0000", 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5533999999999997e-05 2024-09-21T23:18:28.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.730994+0000", 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7767000000000001e-05 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731005+0000", 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0689999999999999e-05 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742143+0000", 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011138171000000001 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742272+0000", 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012899999999999999 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.742477+0000", 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000205162 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744916+0000", 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024387269999999999 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746037+0000", 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001120956 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746117+0000", 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746182+0000", 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5098000000000006e-05 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.79017502599999, 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016753543999999999, 2024-09-21T23:18:28.542 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 113 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731027+0000", 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8700000000000004e-07 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731034+0000", 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3459999999999999e-06 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731044+0000", 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0427e-05 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731053+0000", 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2690000000000005e-06 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743667+0000", 2024-09-21T23:18:28.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01261317 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.743837+0000", 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170252 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.744059+0000", 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222244 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747630+0000", 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035705300000000001 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747652+0000", 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2195999999999999e-05 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747722+0000", 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0055000000000006e-05 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747780+0000", 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8168000000000003e-05 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.79010591100001, 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018030733, 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 114 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731096+0000", 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731097+0000", 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3300000000000001e-07 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731104+0000", 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1799999999999999e-06 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731118+0000", 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.412e-05 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731126+0000", 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.49e-06 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745028+0000", 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013901419999999999 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745198+0000", 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000170259 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.745428+0000", 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230492 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.747880+0000", 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002451323 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748968+0000", 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0010885370000000001 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749062+0000", 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4141000000000005e-05 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749126+0000", 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3937999999999997e-05 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.553 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.78949358599999, 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018589713000000001, 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 115 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731708+0000", 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731710+0000", 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6869999999999999e-06 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731853+0000", 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014305000000000001 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731872+0000", 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8961999999999999e-05 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731887+0000", 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5133000000000001e-05 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746282+0000", 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014395539000000001 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746445+0000", 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016290599999999999 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.746652+0000", 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020631800000000001 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750138+0000", 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034864679999999999 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750159+0000", 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.056e-05 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750234+0000", 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5821000000000005e-05 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750298+0000", 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3268999999999993e-05 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.559 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.78928918099999, 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018725489000000001, 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 116 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731912+0000", 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731913+0000", 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3699999999999999e-07 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731922+0000", 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731933+0000", 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0827e-05 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731940+0000", 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9200000000000004e-06 2024-09-21T23:18:28.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748001+0000", 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016060110999999998 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748167+0000", 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016645900000000001 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.748412+0000", 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024485099999999998 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750317+0000", 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019050130000000001 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750385+0000", 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7679e-05 2024-09-21T23:18:28.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750508+0000", 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012389000000000001 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750638+0000", 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012946299999999999 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.565 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.78924639499999, 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019156271999999998, 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 117 2024-09-21T23:18:28.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731955+0000", 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731956+0000", 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7000000000000004e-07 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731963+0000", 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6529999999999997e-06 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731972+0000", 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7759999999999992e-06 2024-09-21T23:18:28.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.731980+0000", 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0770000000000008e-06 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749199+0000", 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017218135999999998 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749358+0000", 2024-09-21T23:18:28.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000159529 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.749566+0000", 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020795199999999999 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750716+0000", 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011503049999999999 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.750925+0000", 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000208252 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751001+0000", 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6611000000000002e-05 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:46.751111+0000", 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011031099999999999 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.571 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39828810700001, 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013796678999999999, 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 130 2024-09-21T23:18:28.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122913+0000", 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122914+0000", 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7700000000000003e-07 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122920+0000", 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0399999999999998e-06 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122933+0000", 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2485e-05 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.122940+0000", 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5109999999999997e-06 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123815+0000", 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087437799999999998 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123896+0000", 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1595999999999996e-05 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124034+0000", 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000138114 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129487+0000", 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005452564 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136540+0000", 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070533710000000001 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136637+0000", 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6255000000000006e-05 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136710+0000", 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3487999999999998e-05 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.577 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.398201597, 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013937926, 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 131 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123000+0000", 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123001+0000", 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8100000000000002e-07 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123052+0000", 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1523999999999997e-05 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123061+0000", 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.422e-06 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123066+0000", 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0849999999999996e-06 2024-09-21T23:18:28.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124475+0000", 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001409495 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124562+0000", 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6669999999999995e-05 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.124699+0000", 2024-09-21T23:18:28.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013720299999999999 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132160+0000", 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074611369999999996 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136838+0000", 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046775610000000002 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136894+0000", 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5826999999999999e-05 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136938+0000", 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4320999999999998e-05 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.583 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39812470800001, 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109428, 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 132 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123077+0000", 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123078+0000", 2024-09-21T23:18:28.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3e-07 2024-09-21T23:18:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123080+0000", 2024-09-21T23:18:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8150000000000002e-06 2024-09-21T23:18:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123086+0000", 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2499999999999997e-06 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123089+0000", 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.749e-06 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125445+0000", 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023560909999999998 2024-09-21T23:18:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125705+0000", 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025998699999999999 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.125986+0000", 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028027299999999999 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136226+0000", 2024-09-21T23:18:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010239933 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136998+0000", 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00077267199999999996 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137085+0000", 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6293999999999997e-05 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137186+0000", 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000101734 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.397919868, 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014056734, 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 133 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123282+0000", 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123283+0000", 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0690000000000001e-06 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123289+0000", 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9610000000000002e-06 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123300+0000", 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1521e-05 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123308+0000", 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4270000000000002e-06 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126788+0000", 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034801649999999999 2024-09-21T23:18:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.126876+0000", 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8041000000000006e-05 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127015+0000", 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013891399999999999 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136257+0000", 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0092426460000000002 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137241+0000", 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00098393099999999995 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137296+0000", 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4716999999999998e-05 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137338+0000", 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2342000000000002e-05 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.397831586, 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014109020999999999, 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 134 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123370+0000", 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123371+0000", 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7800000000000001e-07 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123460+0000", 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9656999999999999e-05 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123474+0000", 2024-09-21T23:18:28.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3884e-05 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123482+0000", 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5750000000000003e-06 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127404+0000", 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039218619999999999 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127490+0000", 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6048e-05 2024-09-21T23:18:28.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.127624+0000", 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013401400000000001 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136733+0000", 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0091090919999999992 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137384+0000", 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065141799999999996 2024-09-21T23:18:28.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137437+0000", 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2648999999999997e-05 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137479+0000", 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2043999999999999e-05 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39770075800001, 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014117958, 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 135 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.607 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123501+0000", 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123502+0000", 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0900000000000001e-07 2024-09-21T23:18:28.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123507+0000", 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6849999999999998e-06 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123520+0000", 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2418e-05 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123531+0000", 2024-09-21T23:18:28.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1834999999999999e-05 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128366+0000", 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.004834344 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128552+0000", 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018663800000000001 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.128775+0000", 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000223038 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136744+0000", 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0079683719999999996 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137528+0000", 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00078417600000000004 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137578+0000", 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0419e-05 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137619+0000", 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0324000000000002e-05 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.612 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39765510500001, 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014324305000000001, 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 136 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123546+0000", 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123547+0000", 2024-09-21T23:18:28.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5600000000000005e-07 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123555+0000", 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0499999999999992e-06 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123565+0000", 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0295000000000001e-05 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123572+0000", 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.832e-06 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129545+0000", 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059724050000000001 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129631+0000", 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6774000000000006e-05 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.129778+0000", 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014658 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136754+0000", 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069760539999999998 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137711+0000", 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00095652599999999997 2024-09-21T23:18:28.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137803+0000", 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2538000000000001e-05 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137871+0000", 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7595000000000006e-05 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.618 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39761667799999, 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014758676, 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 137 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123585+0000", 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123586+0000", 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0200000000000001e-07 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123679+0000", 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3419999999999997e-05 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123698+0000", 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8556e-05 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123703+0000", 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6890000000000001e-06 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130206+0000", 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065026290000000002 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130348+0000", 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014159599999999999 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.130599+0000", 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025106 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136764+0000", 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00616578 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138251+0000", 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014863089999999999 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138303+0000", 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2496e-05 2024-09-21T23:18:28.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138344+0000", 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0438999999999997e-05 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.397487216, 2024-09-21T23:18:28.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014776447, 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 138 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123714+0000", 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123715+0000", 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6600000000000002e-07 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123718+0000", 2024-09-21T23:18:28.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2499999999999998e-06 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123725+0000", 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3259999999999998e-06 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123730+0000", 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3630000000000004e-06 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132218+0000", 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084883240000000002 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132302+0000", 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4259999999999996e-05 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132444+0000", 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014155400000000001 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136773+0000", 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043287129999999997 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138401+0000", 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001628287 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138448+0000", 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6839999999999999e-05 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138491+0000", 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3064e-05 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39746557399999, 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014912075, 2024-09-21T23:18:28.630 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 139 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123736+0000", 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9999999999999999e-07 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123742+0000", 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6339999999999998e-06 2024-09-21T23:18:28.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123747+0000", 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.622e-06 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123750+0000", 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8500000000000004e-06 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132832+0000", 2024-09-21T23:18:28.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0090816639999999997 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.132917+0000", 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4590000000000002e-05 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133053+0000", 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013658999999999999 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136783+0000", 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003729727 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138571+0000", 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0017882950000000001 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138618+0000", 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6328000000000001e-05 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138648+0000", 2024-09-21T23:18:28.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0474999999999999e-05 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39744553, 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014998997999999999, 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.636 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 140 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123756+0000", 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1899999999999998e-07 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123819+0000", 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2292999999999995e-05 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123825+0000", 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123832+0000", 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.421e-06 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133429+0000", 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0095979340000000007 2024-09-21T23:18:28.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133516+0000", 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6623000000000001e-05 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.133668+0000", 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015193699999999999 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.136793+0000", 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003125306 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138686+0000", 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001892447 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138722+0000", 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5862000000000003e-05 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.138755+0000", 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3408999999999998e-05 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 221.39736153800001, 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015382599, 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "continuous": false, 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T23:18:28.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4177", 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.144:0/282926801", 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 141 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123840+0000", 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-21T23:18:28.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123841+0000", 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3099999999999998e-07 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123845+0000", 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4460000000000003e-06 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123851+0000", 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5140000000000001e-06 2024-09-21T23:18:28.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.123854+0000", 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6500000000000002e-06 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134373+0000", 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010519003000000001 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.134648+0000", 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000274906 2024-09-21T23:18:28.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 0", 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.135078+0000", 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00042982300000000002 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.137895+0000", 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028173109999999999 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139158+0000", 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001263122 2024-09-21T23:18:28.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139195+0000", 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7088999999999997e-05 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-21T23:14:47.139223+0000", 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7203999999999999e-05 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-21T23:18:28.647 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-21T23:18:28.648 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-21T23:18:28.649 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T23:18:28.649 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-21T23:18:28.649 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T23:18:28.650 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:28.650 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-21T23:18:28.650 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T23:18:28.651 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-21T23:18:28.651 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T23:18:28.651 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-21T23:18:28.652 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T23:18:28.652 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:28.650+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:28.750 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:28.750+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:28.824 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:28.824 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:28.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:28.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:18:28.881 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:28.880+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:28.930 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:18:28.951 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-21T23:18:28.948+0000 7f055d89e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25337) UID: 0 2024-09-21T23:18:28.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:18:28.965 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:28.965 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:28.980 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:29.069 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:29.068+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:29.153 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:29.148+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:29.156 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:29.156 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:29.171 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:29.257 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-21T23:18:29.256+0000 7f22434f6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25333) UID: 0 2024-09-21T23:18:29.336 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:29.336 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:29.350 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:29.355 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:29.358+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:29.455 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-21T23:18:29.458+0000 7f75fde1b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25183) UID: 0 2024-09-21T23:18:29.517 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:29.517 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:29.531 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:29.557 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:29.558+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:29.653 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:29.653 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:29.657 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:29.652+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:29.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:29.758 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:29.756+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:29.799 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:29.799 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:29.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:29.859 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:29.862+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:29.933 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:18:29.933 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:18:29.946 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:29.946 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:29.960 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-21T23:18:29.962+0000 7f4719737640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25186) UID: 0 2024-09-21T23:18:29.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:29.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:18:30.061 INFO:tasks.ceph.osd.3.smithi144.stderr:2024-09-21T23:18:30.056+0000 7f9fbe0d6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25334) UID: 0 2024-09-21T23:18:30.129 INFO:teuthology.orchestra.run.smithi170.stdout:All operations completed A-OK! 2024-09-21T23:18:30.139 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:30.139 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:30.154 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T23:18:30.199 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-21T23:18:30.202+0000 7f973ddb9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34314) UID: 0 2024-09-21T23:18:30.208 DEBUG:teuthology.parallel:result is None 2024-09-21T23:18:30.209 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-21T23:18:30.217 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T23:18:30.233 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T23:18:30.265 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:30.265 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:30.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T23:18:30.431 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:30.431 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:30.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T23:18:30.571 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:30.571 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:30.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T23:18:30.712 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:30.712 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:30.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T23:18:30.853 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:30.853 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:30.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T23:18:30.941 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:18:30.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:18:30.997 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:30.997 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.012 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T23:18:31.143 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:31.144 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.158 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T23:18:31.281 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:31.281 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.296 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T23:18:31.419 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:31.420 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.434 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T23:18:31.555 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:31.556 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.570 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T23:18:31.688 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.704 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T23:18:31.775 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:18:31.802 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:18:31.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:31.826 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:31.827 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.841 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T23:18:31.958 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:31.958 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:31.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T23:18:31.972 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T23:18:32.094 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:32.094 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:32.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T23:18:32.123 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:32.124 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:32.142 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-21T23:18:31.767466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3197043,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":330,"num_read":313931,"num_read_kb":1079414,"num_write":21373,"num_write_kb":210620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2134,"num_bytes_recovered":24352285,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20363,"ondisk_log_size":20363,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":246748,"kb_used_data":9780,"kb_used_omap":10,"kb_used_meta":236853,"kb_avail":377240612,"statfs":{"total":386547056640,"available":386294386688,"internally_reserved":0,"allocated":10014720,"data_stored":4974066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":242538100},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-35394020,"num_objects":-119,"num_object_clones":-20,"num_object_copies":-238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-119,"num_whiteouts":-20,"num_read":1145,"num_read_kb":985,"num_write":243,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.849085"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13412,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.949817+0000","last_change":"2024-09-21T23:18:29.949817+0000","last_active":"2024-09-21T23:18:29.949817+0000","last_peered":"2024-09-21T23:18:29.949817+0000","last_clean":"2024-09-21T23:18:29.949817+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:18:29.949817+0000","last_undegraded":"2024-09-21T23:18:29.949817+0000","last_fullsized":"2024-09-21T23:18:29.949817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:57:30.841851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.1e","version":"177'484","reported_seq":5656,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.956385+0000","last_change":"2024-09-21T23:18:28.956873+0000","last_active":"2024-09-21T23:18:28.956385+0000","last_peered":"2024-09-21T23:18:28.956385+0000","last_clean":"2024-09-21T23:18:28.956385+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:18:28.956385+0000","last_undegraded":"2024-09-21T23:18:28.956385+0000","last_fullsized":"2024-09-21T23:18:28.956385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:46.346245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020635269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.1d","version":"177'597","reported_seq":8040,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.938627+0000","last_change":"2024-09-21T23:18:28.939451+0000","last_active":"2024-09-21T23:18:28.938627+0000","last_peered":"2024-09-21T23:18:28.938627+0000","last_clean":"2024-09-21T23:18:28.938627+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:28.938627+0000","last_undegraded":"2024-09-21T23:18:28.938627+0000","last_fullsized":"2024-09-21T23:18:28.938627+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:01.160656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012092769999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.1c","version":"178'1014","reported_seq":13012,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.000735+0000","last_change":"2024-09-21T23:18:28.943617+0000","last_active":"2024-09-21T23:18:29.000735+0000","last_peered":"2024-09-21T23:18:29.000735+0000","last_clean":"2024-09-21T23:18:29.000735+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:29.000735+0000","last_undegraded":"2024-09-21T23:18:29.000735+0000","last_fullsized":"2024-09-21T23:18:29.000735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1014,"log_dups_size":0,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:32:39.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048722160000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17733,"num_read_kb":71143,"num_write":1047,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.1b","version":"179'490","reported_seq":5287,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:30.122203+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:30.122203+0000","last_peered":"2024-09-21T23:18:30.122203+0000","last_clean":"2024-09-21T23:18:30.122203+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:30.122203+0000","last_undegraded":"2024-09-21T23:18:30.122203+0000","last_fullsized":"2024-09-21T23:18:30.122203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.1a","version":"178'696","reported_seq":10476,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.100560+0000","last_change":"2024-09-21T23:18:28.950217+0000","last_active":"2024-09-21T23:18:29.100560+0000","last_peered":"2024-09-21T23:18:29.100560+0000","last_clean":"2024-09-21T23:18:29.100560+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:29.100560+0000","last_undegraded":"2024-09-21T23:18:29.100560+0000","last_fullsized":"2024-09-21T23:18:29.100560+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:42.046039+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011920533000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12153,"num_read_kb":26814,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.19","version":"176'411","reported_seq":5393,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.954878+0000","last_change":"2024-09-21T23:18:28.955254+0000","last_active":"2024-09-21T23:18:28.954878+0000","last_peered":"2024-09-21T23:18:28.954878+0000","last_clean":"2024-09-21T23:18:28.954878+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:28.954878+0000","last_undegraded":"2024-09-21T23:18:28.954878+0000","last_fullsized":"2024-09-21T23:18:28.954878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:34.788239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014617930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.18","version":"178'672","reported_seq":18620,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944602+0000","last_change":"2024-09-21T23:18:29.946267+0000","last_active":"2024-09-21T23:18:29.944602+0000","last_peered":"2024-09-21T23:18:29.944602+0000","last_clean":"2024-09-21T23:18:29.944602+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:18:29.944602+0000","last_undegraded":"2024-09-21T23:18:29.944602+0000","last_fullsized":"2024-09-21T23:18:29.944602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:53:08.598918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.17","version":"178'416","reported_seq":13633,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.947899+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:29.947899+0000","last_peered":"2024-09-21T23:18:29.947899+0000","last_clean":"2024-09-21T23:18:29.947899+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:29.947899+0000","last_undegraded":"2024-09-21T23:18:29.947899+0000","last_fullsized":"2024-09-21T23:18:29.947899+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.16","version":"178'581","reported_seq":5921,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.071522+0000","last_change":"2024-09-21T23:18:28.946543+0000","last_active":"2024-09-21T23:18:29.071522+0000","last_peered":"2024-09-21T23:18:29.071522+0000","last_clean":"2024-09-21T23:18:29.071522+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:18:29.071522+0000","last_undegraded":"2024-09-21T23:18:29.071522+0000","last_fullsized":"2024-09-21T23:18:29.071522+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:54.807712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068013800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.15","version":"177'349","reported_seq":4251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.947804+0000","last_change":"2024-09-21T23:18:29.948897+0000","last_active":"2024-09-21T23:18:29.947804+0000","last_peered":"2024-09-21T23:18:29.947804+0000","last_clean":"2024-09-21T23:18:29.947804+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:18:29.947804+0000","last_undegraded":"2024-09-21T23:18:29.947804+0000","last_fullsized":"2024-09-21T23:18:29.947804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:19:07.037842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.14","version":"178'938","reported_seq":8252,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.952572+0000","last_change":"2024-09-21T23:18:28.953090+0000","last_active":"2024-09-21T23:18:28.952572+0000","last_peered":"2024-09-21T23:18:28.952572+0000","last_clean":"2024-09-21T23:18:28.952572+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:18:28.952572+0000","last_undegraded":"2024-09-21T23:18:28.952572+0000","last_fullsized":"2024-09-21T23:18:28.952572+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:51:32.051376+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011213768000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":321,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.939932+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:18:28.939932+0000","last_peered":"2024-09-21T23:18:28.939932+0000","last_clean":"2024-09-21T23:18:28.939932+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:18:28.939932+0000","last_undegraded":"2024-09-21T23:18:28.939932+0000","last_fullsized":"2024-09-21T23:18:28.939932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.947728+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:29.947728+0000","last_peered":"2024-09-21T23:18:29.947728+0000","last_clean":"2024-09-21T23:18:29.947728+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:29.947728+0000","last_undegraded":"2024-09-21T23:18:29.947728+0000","last_fullsized":"2024-09-21T23:18:29.947728+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.954757+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:18:28.954757+0000","last_peered":"2024-09-21T23:18:28.954757+0000","last_clean":"2024-09-21T23:18:28.954757+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:18:28.954757+0000","last_undegraded":"2024-09-21T23:18:28.954757+0000","last_fullsized":"2024-09-21T23:18:28.954757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8553,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.956954+0000","last_change":"2024-09-21T23:18:28.957230+0000","last_active":"2024-09-21T23:18:28.956954+0000","last_peered":"2024-09-21T23:18:28.956954+0000","last_clean":"2024-09-21T23:18:28.956954+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:18:28.956954+0000","last_undegraded":"2024-09-21T23:18:28.956954+0000","last_fullsized":"2024-09-21T23:18:28.956954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:02:15.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002369599,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944788+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:18:29.944788+0000","last_peered":"2024-09-21T23:18:29.944788+0000","last_clean":"2024-09-21T23:18:29.944788+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:18:29.944788+0000","last_undegraded":"2024-09-21T23:18:29.944788+0000","last_fullsized":"2024-09-21T23:18:29.944788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":2885,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.955350+0000","last_change":"2024-09-21T23:18:28.955714+0000","last_active":"2024-09-21T23:18:28.955350+0000","last_peered":"2024-09-21T23:18:28.955350+0000","last_clean":"2024-09-21T23:18:28.955350+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:18:28.955350+0000","last_undegraded":"2024-09-21T23:18:28.955350+0000","last_fullsized":"2024-09-21T23:18:28.955350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:31:11.065056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016907459999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.953965+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:28.953965+0000","last_peered":"2024-09-21T23:18:28.953965+0000","last_clean":"2024-09-21T23:18:28.953965+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:28.953965+0000","last_undegraded":"2024-09-21T23:18:28.953965+0000","last_fullsized":"2024-09-21T23:18:28.953965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":10734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.943721+0000","last_change":"2024-09-21T23:18:28.944564+0000","last_active":"2024-09-21T23:18:28.943721+0000","last_peered":"2024-09-21T23:18:28.943721+0000","last_clean":"2024-09-21T23:18:28.943721+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:18:28.943721+0000","last_undegraded":"2024-09-21T23:18:28.943721+0000","last_fullsized":"2024-09-21T23:18:28.943721+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:10.680385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056551209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.3","version":"178'788","reported_seq":7063,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.086656+0000","last_change":"2024-09-21T23:18:28.948786+0000","last_active":"2024-09-21T23:18:29.086656+0000","last_peered":"2024-09-21T23:18:29.086656+0000","last_clean":"2024-09-21T23:18:29.086656+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:29.086656+0000","last_undegraded":"2024-09-21T23:18:29.086656+0000","last_fullsized":"2024-09-21T23:18:29.086656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:03.986329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094257720000000007,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8406,"num_read_kb":29127,"num_write":767,"num_write_kb":5626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.2","version":"19'2","reported_seq":360,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.938136+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:28.938136+0000","last_peered":"2024-09-21T23:18:28.938136+0000","last_clean":"2024-09-21T23:18:28.938136+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:28.938136+0000","last_undegraded":"2024-09-21T23:18:28.938136+0000","last_fullsized":"2024-09-21T23:18:28.938136+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6426,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.959697+0000","last_change":"2024-09-21T23:18:28.960210+0000","last_active":"2024-09-21T23:18:28.959697+0000","last_peered":"2024-09-21T23:18:28.959697+0000","last_clean":"2024-09-21T23:18:28.959697+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:18:28.959697+0000","last_undegraded":"2024-09-21T23:18:28.959697+0000","last_fullsized":"2024-09-21T23:18:28.959697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:37.554935+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0061051580000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.953527+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:18:28.953527+0000","last_peered":"2024-09-21T23:18:28.953527+0000","last_clean":"2024-09-21T23:18:28.953527+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:18:28.953527+0000","last_undegraded":"2024-09-21T23:18:28.953527+0000","last_fullsized":"2024-09-21T23:18:28.953527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4751,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944880+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:29.944880+0000","last_peered":"2024-09-21T23:18:29.944880+0000","last_clean":"2024-09-21T23:18:29.944880+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:29.944880+0000","last_undegraded":"2024-09-21T23:18:29.944880+0000","last_fullsized":"2024-09-21T23:18:29.944880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.953917+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:18:28.953917+0000","last_peered":"2024-09-21T23:18:28.953917+0000","last_clean":"2024-09-21T23:18:28.953917+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:18:28.953917+0000","last_undegraded":"2024-09-21T23:18:28.953917+0000","last_fullsized":"2024-09-21T23:18:28.953917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":16982,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.128774+0000","last_change":"2024-09-21T23:18:28.956301+0000","last_active":"2024-09-21T23:18:29.128774+0000","last_peered":"2024-09-21T23:18:29.128774+0000","last_clean":"2024-09-21T23:18:29.128774+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:29.128774+0000","last_undegraded":"2024-09-21T23:18:29.128774+0000","last_fullsized":"2024-09-21T23:18:29.128774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:24.716044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020721060000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.3","version":"17'1","reported_seq":347,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.938464+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:18:28.938464+0000","last_peered":"2024-09-21T23:18:28.938464+0000","last_clean":"2024-09-21T23:18:28.938464+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:18:28.938464+0000","last_undegraded":"2024-09-21T23:18:28.938464+0000","last_fullsized":"2024-09-21T23:18:28.938464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":439,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944228+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:18:29.944228+0000","last_peered":"2024-09-21T23:18:29.944228+0000","last_clean":"2024-09-21T23:18:29.944228+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:18:29.944228+0000","last_undegraded":"2024-09-21T23:18:29.944228+0000","last_fullsized":"2024-09-21T23:18:29.944228+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12880,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.954788+0000","last_change":"2024-09-21T23:18:28.955324+0000","last_active":"2024-09-21T23:18:28.954788+0000","last_peered":"2024-09-21T23:18:28.954788+0000","last_clean":"2024-09-21T23:18:28.954788+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:18:28.954788+0000","last_undegraded":"2024-09-21T23:18:28.954788+0000","last_fullsized":"2024-09-21T23:18:28.954788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:09.255403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001571591,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.9","version":"178'592","reported_seq":5047,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.078229+0000","last_change":"2024-09-21T23:18:28.947687+0000","last_active":"2024-09-21T23:18:29.078229+0000","last_peered":"2024-09-21T23:18:29.078229+0000","last_clean":"2024-09-21T23:18:29.078229+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:18:29.078229+0000","last_undegraded":"2024-09-21T23:18:29.078229+0000","last_fullsized":"2024-09-21T23:18:29.078229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:34:06.368972+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0096518929999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.a","version":"178'354","reported_seq":4223,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.942405+0000","last_change":"2024-09-21T23:18:28.942405+0000","last_active":"2024-09-21T23:18:28.942405+0000","last_peered":"2024-09-21T23:18:28.942405+0000","last_clean":"2024-09-21T23:18:28.942405+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:18:28.942405+0000","last_undegraded":"2024-09-21T23:18:28.942405+0000","last_fullsized":"2024-09-21T23:18:28.942405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:24:21.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.b","version":"177'791","reported_seq":8628,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.949065+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:29.949065+0000","last_peered":"2024-09-21T23:18:29.949065+0000","last_clean":"2024-09-21T23:18:29.949065+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:29.949065+0000","last_undegraded":"2024-09-21T23:18:29.949065+0000","last_fullsized":"2024-09-21T23:18:29.949065+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.c","version":"178'676","reported_seq":6108,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.121589+0000","last_change":"2024-09-21T23:18:28.944884+0000","last_active":"2024-09-21T23:18:29.121589+0000","last_peered":"2024-09-21T23:18:29.121589+0000","last_clean":"2024-09-21T23:18:29.121589+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:18:29.121589+0000","last_undegraded":"2024-09-21T23:18:29.121589+0000","last_fullsized":"2024-09-21T23:18:29.121589+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:42:53.802459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059394410000000002,"stat_sum":{"num_bytes":45,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7951,"num_read_kb":18218,"num_write":504,"num_write_kb":3353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.d","version":"177'475","reported_seq":5367,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.948880+0000","last_change":"2024-09-21T23:18:28.949681+0000","last_active":"2024-09-21T23:18:28.948880+0000","last_peered":"2024-09-21T23:18:28.948880+0000","last_clean":"2024-09-21T23:18:28.948880+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:18:28.948880+0000","last_undegraded":"2024-09-21T23:18:28.948880+0000","last_fullsized":"2024-09-21T23:18:28.948880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:24:00.164431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010296507,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.e","version":"173'472","reported_seq":4570,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.944978+0000","last_change":"2024-09-21T23:18:28.945456+0000","last_active":"2024-09-21T23:18:28.944978+0000","last_peered":"2024-09-21T23:18:28.944978+0000","last_clean":"2024-09-21T23:18:28.944978+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:28.944978+0000","last_undegraded":"2024-09-21T23:18:28.944978+0000","last_fullsized":"2024-09-21T23:18:28.944978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:28.507698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058303249999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.f","version":"176'479","reported_seq":6610,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.945558+0000","last_change":"2024-09-21T23:18:28.946065+0000","last_active":"2024-09-21T23:18:28.945558+0000","last_peered":"2024-09-21T23:18:28.945558+0000","last_clean":"2024-09-21T23:18:28.945558+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:18:28.945558+0000","last_undegraded":"2024-09-21T23:18:28.945558+0000","last_fullsized":"2024-09-21T23:18:28.945558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:28.359622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055863249999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.10","version":"178'776","reported_seq":6150,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.943843+0000","last_change":"2024-09-21T23:18:28.944394+0000","last_active":"2024-09-21T23:18:28.943843+0000","last_peered":"2024-09-21T23:18:28.943843+0000","last_clean":"2024-09-21T23:18:28.943843+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:28.943843+0000","last_undegraded":"2024-09-21T23:18:28.943843+0000","last_fullsized":"2024-09-21T23:18:28.943843+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:04:57.603423+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0065909059999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.11","version":"178'539","reported_seq":5542,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.945012+0000","last_change":"2024-09-21T23:18:28.946033+0000","last_active":"2024-09-21T23:18:28.945012+0000","last_peered":"2024-09-21T23:18:28.945012+0000","last_clean":"2024-09-21T23:18:28.945012+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:18:28.945012+0000","last_undegraded":"2024-09-21T23:18:28.945012+0000","last_fullsized":"2024-09-21T23:18:28.945012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:51:42.295698+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084505069999999995,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.12","version":"177'807","reported_seq":22063,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.946140+0000","last_change":"2024-09-21T23:18:29.947694+0000","last_active":"2024-09-21T23:18:29.946140+0000","last_peered":"2024-09-21T23:18:29.946140+0000","last_clean":"2024-09-21T23:18:29.946140+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:18:29.946140+0000","last_undegraded":"2024-09-21T23:18:29.946140+0000","last_fullsized":"2024-09-21T23:18:29.946140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:16.175468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.13","version":"176'483","reported_seq":9785,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.941895+0000","last_change":"2024-09-21T23:18:28.943042+0000","last_active":"2024-09-21T23:18:28.941895+0000","last_peered":"2024-09-21T23:18:28.941895+0000","last_clean":"2024-09-21T23:18:28.941895+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:18:28.941895+0000","last_undegraded":"2024-09-21T23:18:28.941895+0000","last_fullsized":"2024-09-21T23:18:28.941895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:24.644787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046787779999999998,"stat_sum":{"num_bytes":2606592,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":526,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2606656,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":330,"num_read":313815,"num_read_kb":1079314,"num_write":21212,"num_write_kb":208202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":720896,"data_stored":624403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20243,"ondisk_log_size":20243,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607601,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51456,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320384,"statfs":{"total":96636764160,"available":96584073216,"internally_reserved":0,"allocated":2269184,"data_stored":1001275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2260992,"data_stored":1001214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78296,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293544,"statfs":{"total":96636764160,"available":96556589056,"internally_reserved":0,"allocated":2744320,"data_stored":1483137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61392,"kb_used_data":2676,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310448,"statfs":{"total":96636764160,"available":96573898752,"internally_reserved":0,"allocated":2740224,"data_stored":1488440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":108563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":108605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:32.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:32.236 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:32.236 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:32.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T23:18:32.367 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:32.367 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:32.380 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T23:18:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:32.450 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:32.466 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-21T23:18:31.767466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3197043,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":330,"num_read":313931,"num_read_kb":1079414,"num_write":21373,"num_write_kb":210620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2134,"num_bytes_recovered":24352285,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20363,"ondisk_log_size":20363,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":246748,"kb_used_data":9780,"kb_used_omap":10,"kb_used_meta":236853,"kb_avail":377240612,"statfs":{"total":386547056640,"available":386294386688,"internally_reserved":0,"allocated":10014720,"data_stored":4974066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10636,"internal_metadata":242538100},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-35394020,"num_objects":-119,"num_object_clones":-20,"num_object_copies":-238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-119,"num_whiteouts":-20,"num_read":1145,"num_read_kb":985,"num_write":243,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.849085"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13412,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.949817+0000","last_change":"2024-09-21T23:18:29.949817+0000","last_active":"2024-09-21T23:18:29.949817+0000","last_peered":"2024-09-21T23:18:29.949817+0000","last_clean":"2024-09-21T23:18:29.949817+0000","last_became_active":"2024-09-21T23:16:36.050260+0000","last_became_peered":"2024-09-21T23:16:36.050260+0000","last_unstale":"2024-09-21T23:18:29.949817+0000","last_undegraded":"2024-09-21T23:18:29.949817+0000","last_fullsized":"2024-09-21T23:18:29.949817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:57:30.841851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.1e","version":"177'484","reported_seq":5656,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.956385+0000","last_change":"2024-09-21T23:18:28.956873+0000","last_active":"2024-09-21T23:18:28.956385+0000","last_peered":"2024-09-21T23:18:28.956385+0000","last_clean":"2024-09-21T23:18:28.956385+0000","last_became_active":"2024-09-21T23:16:36.048084+0000","last_became_peered":"2024-09-21T23:16:36.048084+0000","last_unstale":"2024-09-21T23:18:28.956385+0000","last_undegraded":"2024-09-21T23:18:28.956385+0000","last_fullsized":"2024-09-21T23:18:28.956385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:46.346245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020635269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.1d","version":"177'597","reported_seq":8040,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.938627+0000","last_change":"2024-09-21T23:18:28.939451+0000","last_active":"2024-09-21T23:18:28.938627+0000","last_peered":"2024-09-21T23:18:28.938627+0000","last_clean":"2024-09-21T23:18:28.938627+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:28.938627+0000","last_undegraded":"2024-09-21T23:18:28.938627+0000","last_fullsized":"2024-09-21T23:18:28.938627+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:01.160656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012092769999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.1c","version":"178'1014","reported_seq":13012,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.000735+0000","last_change":"2024-09-21T23:18:28.943617+0000","last_active":"2024-09-21T23:18:29.000735+0000","last_peered":"2024-09-21T23:18:29.000735+0000","last_clean":"2024-09-21T23:18:29.000735+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:29.000735+0000","last_undegraded":"2024-09-21T23:18:29.000735+0000","last_fullsized":"2024-09-21T23:18:29.000735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1014,"log_dups_size":0,"ondisk_log_size":1014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:32:39.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048722160000000004,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17733,"num_read_kb":71143,"num_write":1047,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.1b","version":"179'490","reported_seq":5287,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:30.122203+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:30.122203+0000","last_peered":"2024-09-21T23:18:30.122203+0000","last_clean":"2024-09-21T23:18:30.122203+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:30.122203+0000","last_undegraded":"2024-09-21T23:18:30.122203+0000","last_fullsized":"2024-09-21T23:18:30.122203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.1a","version":"178'696","reported_seq":10476,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.100560+0000","last_change":"2024-09-21T23:18:28.950217+0000","last_active":"2024-09-21T23:18:29.100560+0000","last_peered":"2024-09-21T23:18:29.100560+0000","last_clean":"2024-09-21T23:18:29.100560+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:29.100560+0000","last_undegraded":"2024-09-21T23:18:29.100560+0000","last_fullsized":"2024-09-21T23:18:29.100560+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:42.046039+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011920533000000001,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12153,"num_read_kb":26814,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.19","version":"176'411","reported_seq":5393,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.954878+0000","last_change":"2024-09-21T23:18:28.955254+0000","last_active":"2024-09-21T23:18:28.954878+0000","last_peered":"2024-09-21T23:18:28.954878+0000","last_clean":"2024-09-21T23:18:28.954878+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:28.954878+0000","last_undegraded":"2024-09-21T23:18:28.954878+0000","last_fullsized":"2024-09-21T23:18:28.954878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:34.788239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014617930000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.18","version":"178'672","reported_seq":18620,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944602+0000","last_change":"2024-09-21T23:18:29.946267+0000","last_active":"2024-09-21T23:18:29.944602+0000","last_peered":"2024-09-21T23:18:29.944602+0000","last_clean":"2024-09-21T23:18:29.944602+0000","last_became_active":"2024-09-21T23:16:36.045033+0000","last_became_peered":"2024-09-21T23:16:36.045033+0000","last_unstale":"2024-09-21T23:18:29.944602+0000","last_undegraded":"2024-09-21T23:18:29.944602+0000","last_fullsized":"2024-09-21T23:18:29.944602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:53:08.598918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.17","version":"178'416","reported_seq":13633,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.947899+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:29.947899+0000","last_peered":"2024-09-21T23:18:29.947899+0000","last_clean":"2024-09-21T23:18:29.947899+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:29.947899+0000","last_undegraded":"2024-09-21T23:18:29.947899+0000","last_fullsized":"2024-09-21T23:18:29.947899+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.16","version":"178'581","reported_seq":5921,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.071522+0000","last_change":"2024-09-21T23:18:28.946543+0000","last_active":"2024-09-21T23:18:29.071522+0000","last_peered":"2024-09-21T23:18:29.071522+0000","last_clean":"2024-09-21T23:18:29.071522+0000","last_became_active":"2024-09-21T23:16:18.768019+0000","last_became_peered":"2024-09-21T23:16:18.768019+0000","last_unstale":"2024-09-21T23:18:29.071522+0000","last_undegraded":"2024-09-21T23:18:29.071522+0000","last_fullsized":"2024-09-21T23:18:29.071522+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:54.807712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068013800000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.15","version":"177'349","reported_seq":4251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.947804+0000","last_change":"2024-09-21T23:18:29.948897+0000","last_active":"2024-09-21T23:18:29.947804+0000","last_peered":"2024-09-21T23:18:29.947804+0000","last_clean":"2024-09-21T23:18:29.947804+0000","last_became_active":"2024-09-21T23:16:06.213271+0000","last_became_peered":"2024-09-21T23:16:06.213271+0000","last_unstale":"2024-09-21T23:18:29.947804+0000","last_undegraded":"2024-09-21T23:18:29.947804+0000","last_fullsized":"2024-09-21T23:18:29.947804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:19:07.037842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.14","version":"178'938","reported_seq":8252,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.952572+0000","last_change":"2024-09-21T23:18:28.953090+0000","last_active":"2024-09-21T23:18:28.952572+0000","last_peered":"2024-09-21T23:18:28.952572+0000","last_clean":"2024-09-21T23:18:28.952572+0000","last_became_active":"2024-09-21T23:15:06.352936+0000","last_became_peered":"2024-09-21T23:15:06.352936+0000","last_unstale":"2024-09-21T23:18:28.952572+0000","last_undegraded":"2024-09-21T23:18:28.952572+0000","last_fullsized":"2024-09-21T23:18:28.952572+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:51:32.051376+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011213768000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":321,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.939932+0000","last_change":"2024-09-21T23:16:33.012682+0000","last_active":"2024-09-21T23:18:28.939932+0000","last_peered":"2024-09-21T23:18:28.939932+0000","last_clean":"2024-09-21T23:18:28.939932+0000","last_became_active":"2024-09-21T23:16:33.012029+0000","last_became_peered":"2024-09-21T23:16:33.012029+0000","last_unstale":"2024-09-21T23:18:28.939932+0000","last_undegraded":"2024-09-21T23:18:28.939932+0000","last_fullsized":"2024-09-21T23:18:28.939932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:49:32.612437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.947728+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:29.947728+0000","last_peered":"2024-09-21T23:18:29.947728+0000","last_clean":"2024-09-21T23:18:29.947728+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:29.947728+0000","last_undegraded":"2024-09-21T23:18:29.947728+0000","last_fullsized":"2024-09-21T23:18:29.947728+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.954757+0000","last_change":"2024-09-21T23:16:15.259832+0000","last_active":"2024-09-21T23:18:28.954757+0000","last_peered":"2024-09-21T23:18:28.954757+0000","last_clean":"2024-09-21T23:18:28.954757+0000","last_became_active":"2024-09-21T23:16:15.259543+0000","last_became_peered":"2024-09-21T23:16:15.259543+0000","last_unstale":"2024-09-21T23:18:28.954757+0000","last_undegraded":"2024-09-21T23:18:28.954757+0000","last_fullsized":"2024-09-21T23:18:28.954757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:14:18.534708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8553,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.956954+0000","last_change":"2024-09-21T23:18:28.957230+0000","last_active":"2024-09-21T23:18:28.956954+0000","last_peered":"2024-09-21T23:18:28.956954+0000","last_clean":"2024-09-21T23:18:28.956954+0000","last_became_active":"2024-09-21T23:16:36.051609+0000","last_became_peered":"2024-09-21T23:16:36.051609+0000","last_unstale":"2024-09-21T23:18:28.956954+0000","last_undegraded":"2024-09-21T23:18:28.956954+0000","last_fullsized":"2024-09-21T23:18:28.956954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:02:15.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002369599,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944788+0000","last_change":"2024-09-21T23:16:03.172757+0000","last_active":"2024-09-21T23:18:29.944788+0000","last_peered":"2024-09-21T23:18:29.944788+0000","last_clean":"2024-09-21T23:18:29.944788+0000","last_became_active":"2024-09-21T23:16:03.171533+0000","last_became_peered":"2024-09-21T23:16:03.171533+0000","last_unstale":"2024-09-21T23:18:29.944788+0000","last_undegraded":"2024-09-21T23:18:29.944788+0000","last_fullsized":"2024-09-21T23:18:29.944788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:39:16.244621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":2885,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.955350+0000","last_change":"2024-09-21T23:18:28.955714+0000","last_active":"2024-09-21T23:18:28.955350+0000","last_peered":"2024-09-21T23:18:28.955350+0000","last_clean":"2024-09-21T23:18:28.955350+0000","last_became_active":"2024-09-21T23:16:36.045431+0000","last_became_peered":"2024-09-21T23:16:36.045431+0000","last_unstale":"2024-09-21T23:18:28.955350+0000","last_undegraded":"2024-09-21T23:18:28.955350+0000","last_fullsized":"2024-09-21T23:18:28.955350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:31:11.065056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016907459999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.953965+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:28.953965+0000","last_peered":"2024-09-21T23:18:28.953965+0000","last_clean":"2024-09-21T23:18:28.953965+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:28.953965+0000","last_undegraded":"2024-09-21T23:18:28.953965+0000","last_fullsized":"2024-09-21T23:18:28.953965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":10734,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.943721+0000","last_change":"2024-09-21T23:18:28.944564+0000","last_active":"2024-09-21T23:18:28.943721+0000","last_peered":"2024-09-21T23:18:28.943721+0000","last_clean":"2024-09-21T23:18:28.943721+0000","last_became_active":"2024-09-21T23:16:54.782206+0000","last_became_peered":"2024-09-21T23:16:54.782206+0000","last_unstale":"2024-09-21T23:18:28.943721+0000","last_undegraded":"2024-09-21T23:18:28.943721+0000","last_fullsized":"2024-09-21T23:18:28.943721+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:10.680385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056551209999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.3","version":"178'788","reported_seq":7063,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.086656+0000","last_change":"2024-09-21T23:18:28.948786+0000","last_active":"2024-09-21T23:18:29.086656+0000","last_peered":"2024-09-21T23:18:29.086656+0000","last_clean":"2024-09-21T23:18:29.086656+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:29.086656+0000","last_undegraded":"2024-09-21T23:18:29.086656+0000","last_fullsized":"2024-09-21T23:18:29.086656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:03.986329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094257720000000007,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8406,"num_read_kb":29127,"num_write":767,"num_write_kb":5626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.2","version":"19'2","reported_seq":360,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.938136+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:28.938136+0000","last_peered":"2024-09-21T23:18:28.938136+0000","last_clean":"2024-09-21T23:18:28.938136+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:28.938136+0000","last_undegraded":"2024-09-21T23:18:28.938136+0000","last_fullsized":"2024-09-21T23:18:28.938136+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6426,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.959697+0000","last_change":"2024-09-21T23:18:28.960210+0000","last_active":"2024-09-21T23:18:28.959697+0000","last_peered":"2024-09-21T23:18:28.959697+0000","last_clean":"2024-09-21T23:18:28.959697+0000","last_became_active":"2024-09-21T23:16:36.050166+0000","last_became_peered":"2024-09-21T23:16:36.050166+0000","last_unstale":"2024-09-21T23:18:28.959697+0000","last_undegraded":"2024-09-21T23:18:28.959697+0000","last_fullsized":"2024-09-21T23:18:28.959697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:37.554935+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0061051580000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.953527+0000","last_change":"2024-09-21T23:16:33.014260+0000","last_active":"2024-09-21T23:18:28.953527+0000","last_peered":"2024-09-21T23:18:28.953527+0000","last_clean":"2024-09-21T23:18:28.953527+0000","last_became_active":"2024-09-21T23:16:33.013973+0000","last_became_peered":"2024-09-21T23:16:33.013973+0000","last_unstale":"2024-09-21T23:18:28.953527+0000","last_undegraded":"2024-09-21T23:18:28.953527+0000","last_fullsized":"2024-09-21T23:18:28.953527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:46.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4751,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944880+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:29.944880+0000","last_peered":"2024-09-21T23:18:29.944880+0000","last_clean":"2024-09-21T23:18:29.944880+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:29.944880+0000","last_undegraded":"2024-09-21T23:18:29.944880+0000","last_fullsized":"2024-09-21T23:18:29.944880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":225,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.953917+0000","last_change":"2024-09-21T23:16:15.259490+0000","last_active":"2024-09-21T23:18:28.953917+0000","last_peered":"2024-09-21T23:18:28.953917+0000","last_clean":"2024-09-21T23:18:28.953917+0000","last_became_active":"2024-09-21T23:16:15.259214+0000","last_became_peered":"2024-09-21T23:16:15.259214+0000","last_unstale":"2024-09-21T23:18:28.953917+0000","last_undegraded":"2024-09-21T23:18:28.953917+0000","last_fullsized":"2024-09-21T23:18:28.953917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:42:25.186701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":16982,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.128774+0000","last_change":"2024-09-21T23:18:28.956301+0000","last_active":"2024-09-21T23:18:29.128774+0000","last_peered":"2024-09-21T23:18:29.128774+0000","last_clean":"2024-09-21T23:18:29.128774+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:29.128774+0000","last_undegraded":"2024-09-21T23:18:29.128774+0000","last_fullsized":"2024-09-21T23:18:29.128774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:27:24.716044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020721060000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"2.3","version":"17'1","reported_seq":347,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.938464+0000","last_change":"2024-09-21T23:16:33.017225+0000","last_active":"2024-09-21T23:18:28.938464+0000","last_peered":"2024-09-21T23:18:28.938464+0000","last_clean":"2024-09-21T23:18:28.938464+0000","last_became_active":"2024-09-21T23:16:33.010687+0000","last_became_peered":"2024-09-21T23:16:33.010687+0000","last_unstale":"2024-09-21T23:18:28.938464+0000","last_undegraded":"2024-09-21T23:18:28.938464+0000","last_fullsized":"2024-09-21T23:18:28.938464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:41:48.711627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":439,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.944228+0000","last_change":"2024-09-21T23:16:18.766897+0000","last_active":"2024-09-21T23:18:29.944228+0000","last_peered":"2024-09-21T23:18:29.944228+0000","last_clean":"2024-09-21T23:18:29.944228+0000","last_became_active":"2024-09-21T23:16:18.766346+0000","last_became_peered":"2024-09-21T23:16:18.766346+0000","last_unstale":"2024-09-21T23:18:29.944228+0000","last_undegraded":"2024-09-21T23:18:29.944228+0000","last_fullsized":"2024-09-21T23:18:29.944228+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:35.114949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12880,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.954788+0000","last_change":"2024-09-21T23:18:28.955324+0000","last_active":"2024-09-21T23:18:28.954788+0000","last_peered":"2024-09-21T23:18:28.954788+0000","last_clean":"2024-09-21T23:18:28.954788+0000","last_became_active":"2024-09-21T23:16:36.046627+0000","last_became_peered":"2024-09-21T23:16:36.046627+0000","last_unstale":"2024-09-21T23:18:28.954788+0000","last_undegraded":"2024-09-21T23:18:28.954788+0000","last_fullsized":"2024-09-21T23:18:28.954788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:59:09.255403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001571591,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.9","version":"178'592","reported_seq":5047,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.078229+0000","last_change":"2024-09-21T23:18:28.947687+0000","last_active":"2024-09-21T23:18:29.078229+0000","last_peered":"2024-09-21T23:18:29.078229+0000","last_clean":"2024-09-21T23:18:29.078229+0000","last_became_active":"2024-09-21T23:16:36.047468+0000","last_became_peered":"2024-09-21T23:16:36.047468+0000","last_unstale":"2024-09-21T23:18:29.078229+0000","last_undegraded":"2024-09-21T23:18:29.078229+0000","last_fullsized":"2024-09-21T23:18:29.078229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:34:06.368972+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0096518929999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.a","version":"178'354","reported_seq":4223,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.942405+0000","last_change":"2024-09-21T23:18:28.942405+0000","last_active":"2024-09-21T23:18:28.942405+0000","last_peered":"2024-09-21T23:18:28.942405+0000","last_clean":"2024-09-21T23:18:28.942405+0000","last_became_active":"2024-09-21T23:16:18.774838+0000","last_became_peered":"2024-09-21T23:16:18.774838+0000","last_unstale":"2024-09-21T23:18:28.942405+0000","last_undegraded":"2024-09-21T23:18:28.942405+0000","last_fullsized":"2024-09-21T23:18:28.942405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:24:21.561396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.b","version":"177'791","reported_seq":8628,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.949065+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:29.949065+0000","last_peered":"2024-09-21T23:18:29.949065+0000","last_clean":"2024-09-21T23:18:29.949065+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:29.949065+0000","last_undegraded":"2024-09-21T23:18:29.949065+0000","last_fullsized":"2024-09-21T23:18:29.949065+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.c","version":"178'676","reported_seq":6108,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.121589+0000","last_change":"2024-09-21T23:18:28.944884+0000","last_active":"2024-09-21T23:18:29.121589+0000","last_peered":"2024-09-21T23:18:29.121589+0000","last_clean":"2024-09-21T23:18:29.121589+0000","last_became_active":"2024-09-21T23:16:06.209726+0000","last_became_peered":"2024-09-21T23:16:06.209726+0000","last_unstale":"2024-09-21T23:18:29.121589+0000","last_undegraded":"2024-09-21T23:18:29.121589+0000","last_fullsized":"2024-09-21T23:18:29.121589+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:42:53.802459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059394410000000002,"stat_sum":{"num_bytes":45,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7951,"num_read_kb":18218,"num_write":504,"num_write_kb":3353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.d","version":"177'475","reported_seq":5367,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.948880+0000","last_change":"2024-09-21T23:18:28.949681+0000","last_active":"2024-09-21T23:18:28.948880+0000","last_peered":"2024-09-21T23:18:28.948880+0000","last_clean":"2024-09-21T23:18:28.948880+0000","last_became_active":"2024-09-21T23:16:36.049759+0000","last_became_peered":"2024-09-21T23:16:36.049759+0000","last_unstale":"2024-09-21T23:18:28.948880+0000","last_undegraded":"2024-09-21T23:18:28.948880+0000","last_fullsized":"2024-09-21T23:18:28.948880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:24:00.164431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010296507,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.e","version":"173'472","reported_seq":4570,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.944978+0000","last_change":"2024-09-21T23:18:28.945456+0000","last_active":"2024-09-21T23:18:28.944978+0000","last_peered":"2024-09-21T23:18:28.944978+0000","last_clean":"2024-09-21T23:18:28.944978+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:28.944978+0000","last_undegraded":"2024-09-21T23:18:28.944978+0000","last_fullsized":"2024-09-21T23:18:28.944978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:28.507698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058303249999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.f","version":"176'479","reported_seq":6610,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.945558+0000","last_change":"2024-09-21T23:18:28.946065+0000","last_active":"2024-09-21T23:18:28.945558+0000","last_peered":"2024-09-21T23:18:28.945558+0000","last_clean":"2024-09-21T23:18:28.945558+0000","last_became_active":"2024-09-21T23:16:06.211921+0000","last_became_peered":"2024-09-21T23:16:06.211921+0000","last_unstale":"2024-09-21T23:18:28.945558+0000","last_undegraded":"2024-09-21T23:18:28.945558+0000","last_fullsized":"2024-09-21T23:18:28.945558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:52:28.359622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055863249999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.10","version":"178'776","reported_seq":6150,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.943843+0000","last_change":"2024-09-21T23:18:28.944394+0000","last_active":"2024-09-21T23:18:28.943843+0000","last_peered":"2024-09-21T23:18:28.943843+0000","last_clean":"2024-09-21T23:18:28.943843+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:28.943843+0000","last_undegraded":"2024-09-21T23:18:28.943843+0000","last_fullsized":"2024-09-21T23:18:28.943843+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:04:57.603423+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0065909059999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.11","version":"178'539","reported_seq":5542,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.945012+0000","last_change":"2024-09-21T23:18:28.946033+0000","last_active":"2024-09-21T23:18:28.945012+0000","last_peered":"2024-09-21T23:18:28.945012+0000","last_clean":"2024-09-21T23:18:28.945012+0000","last_became_active":"2024-09-21T23:16:06.214406+0000","last_became_peered":"2024-09-21T23:16:06.214406+0000","last_unstale":"2024-09-21T23:18:28.945012+0000","last_undegraded":"2024-09-21T23:18:28.945012+0000","last_fullsized":"2024-09-21T23:18:28.945012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:51:42.295698+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084505069999999995,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"9"}]},{"pgid":"3.12","version":"177'807","reported_seq":22063,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T23:18:29.946140+0000","last_change":"2024-09-21T23:18:29.947694+0000","last_active":"2024-09-21T23:18:29.946140+0000","last_peered":"2024-09-21T23:18:29.946140+0000","last_clean":"2024-09-21T23:18:29.946140+0000","last_became_active":"2024-09-21T23:16:18.767994+0000","last_became_peered":"2024-09-21T23:16:18.767994+0000","last_unstale":"2024-09-21T23:18:29.946140+0000","last_undegraded":"2024-09-21T23:18:29.946140+0000","last_fullsized":"2024-09-21T23:18:29.946140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:16.175468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"a"}]},{"pgid":"3.13","version":"176'483","reported_seq":9785,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T23:18:28.941895+0000","last_change":"2024-09-21T23:18:28.943042+0000","last_active":"2024-09-21T23:18:28.941895+0000","last_peered":"2024-09-21T23:18:28.941895+0000","last_clean":"2024-09-21T23:18:28.941895+0000","last_became_active":"2024-09-21T23:16:36.048242+0000","last_became_peered":"2024-09-21T23:16:36.048242+0000","last_unstale":"2024-09-21T23:18:28.941895+0000","last_undegraded":"2024-09-21T23:18:28.941895+0000","last_fullsized":"2024-09-21T23:18:28.941895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:14:24.644787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046787779999999998,"stat_sum":{"num_bytes":2606592,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":526,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2606656,"num_objects":336,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":330,"num_read":313815,"num_read_kb":1079314,"num_write":21212,"num_write_kb":208202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":720896,"data_stored":624403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20243,"ondisk_log_size":20243,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607601,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51456,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320384,"statfs":{"total":96636764160,"available":96584073216,"internally_reserved":0,"allocated":2269184,"data_stored":1001275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2260992,"data_stored":1001214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":51539607601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78296,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293544,"statfs":{"total":96636764160,"available":96556589056,"internally_reserved":0,"allocated":2744320,"data_stored":1483137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61392,"kb_used_data":2676,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310448,"statfs":{"total":96636764160,"available":96573898752,"internally_reserved":0,"allocated":2740224,"data_stored":1488440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":407200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":108563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":108605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:32.468 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:18:32.468 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T23:18:32.469 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-21T23:18:32.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-21T23:18:32.492 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:32.492 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:32.505 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T23:18:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-21T23:18:32.678 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T23:18:32.778 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-21T23:18:32.778+0000 7faab395a640 -1 mon.a@0(leader).osd e181 definitely_dead 0 2024-09-21T23:18:32.957 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.3. 2024-09-21T23:18:32.966 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-21T23:18:32.993 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-21T23:18:32.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T23:18:33.966 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-21T23:18:37.994 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-21T23:18:37.994 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-21T23:18:37.994 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-21T23:18:38.323 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-21T23:18:38.326+0000 7faab395a640 -1 mon.a@0(leader).osd e187 definitely_dead 0 2024-09-21T23:18:38.786 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.2. 2024-09-21T23:18:38.813 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-21T23:18:38.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T23:18:38.962 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a: {} 2024-09-21T23:18:38.963 INFO:teuthology.orchestra.run.smithi053.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:18:38.977 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b: {} 2024-09-21T23:18:38.977 INFO:teuthology.orchestra.run.smithi053.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:18:38.989 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c: {} 2024-09-21T23:18:38.989 INFO:teuthology.orchestra.run.smithi053.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T23:18:39.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T23:18:39.787 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T23:18:39.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T23:18:40.789 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T23:18:40.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T23:18:41.771 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T23:18:41.803 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T23:18:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T23:18:42.806 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T23:18:43.781 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T23:18:43.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T23:18:45.002 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T23:18:45.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T23:18:46.011 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T23:18:46.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T23:18:46.778 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T23:18:46.810 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T23:18:47.779 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T23:18:47.816 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T23:18:48.787 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T23:18:48.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T23:18:50.013 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T23:18:50.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T23:18:51.023 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T23:18:51.052 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T23:18:51.052 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T23:18:51.052 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T23:18:51.052 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T23:18:51.052 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T23:18:51.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:18:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:51.357 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":200,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:18:51.024572+0000","last_up_change":"2024-09-21T23:16:02.149456+0000","last_in_change":"2024-09-21T23:18:38.779988+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:14:44.926374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:14:49.598603+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T23:14:59.367341+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":42,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":29,"up_from":61,"up_thru":195,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6817","nonce":1028371170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6819","nonce":1028371170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6823","nonce":1028371170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6821","nonce":1028371170}]},"public_addr":"172.21.15.53:6817/1028371170","cluster_addr":"172.21.15.53:6819/1028371170","heartbeat_back_addr":"172.21.15.53:6823/1028371170","heartbeat_front_addr":"172.21.15.53:6821/1028371170","state":["exists","up"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6809","nonce":2712042978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6811","nonce":2712042978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6815","nonce":2712042978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6813","nonce":2712042978}]},"public_addr":"172.21.15.53:6809/2712042978","cluster_addr":"172.21.15.53:6811/2712042978","heartbeat_back_addr":"172.21.15.53:6815/2712042978","heartbeat_front_addr":"172.21.15.53:6813/2712042978","state":["exists","up"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6801","nonce":1166517858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6803","nonce":1166517858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6807","nonce":1166517858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6805","nonce":1166517858}]},"public_addr":"172.21.15.53:6801/1166517858","cluster_addr":"172.21.15.53:6803/1166517858","heartbeat_back_addr":"172.21.15.53:6807/1166517858","heartbeat_front_addr":"172.21.15.53:6805/1166517858","state":["exists","up"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6811","nonce":1039392120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6816","nonce":1039392120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6823","nonce":1039392120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6821","nonce":1039392120}]},"public_addr":"172.21.15.144:6811/1039392120","cluster_addr":"172.21.15.144:6816/1039392120","heartbeat_back_addr":"172.21.15.144:6823/1039392120","heartbeat_front_addr":"172.21.15.144:6821/1039392120","state":["exists","up"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6810","nonce":2764095640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6813","nonce":2764095640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6820","nonce":2764095640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6817","nonce":2764095640}]},"public_addr":"172.21.15.144:6810/2764095640","cluster_addr":"172.21.15.144:6813/2764095640","heartbeat_back_addr":"172.21.15.144:6820/2764095640","heartbeat_front_addr":"172.21.15.144:6817/2764095640","state":["exists","up"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6801","nonce":1732973431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6803","nonce":1732973431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6807","nonce":1732973431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6805","nonce":1732973431}]},"public_addr":"172.21.15.144:6801/1732973431","cluster_addr":"172.21.15.144:6803/1732973431","heartbeat_back_addr":"172.21.15.144:6807/1732973431","heartbeat_front_addr":"172.21.15.144:6805/1732973431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.468749+0000","dead_epoch":31},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.044837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.310970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.241402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.877818+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.434052+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.b","osds":[0,4]},{"pgid":"3.c","osds":[1,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:18:51.377 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T23:18:51.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:18:51.664 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:51.664 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":200,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:18:51.024572+0000","last_up_change":"2024-09-21T23:16:02.149456+0000","last_in_change":"2024-09-21T23:18:38.779988+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:14:44.926374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:14:49.598603+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T23:14:59.367341+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":42,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":29,"up_from":61,"up_thru":195,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6817","nonce":1028371170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6819","nonce":1028371170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6823","nonce":1028371170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6821","nonce":1028371170}]},"public_addr":"172.21.15.53:6817/1028371170","cluster_addr":"172.21.15.53:6819/1028371170","heartbeat_back_addr":"172.21.15.53:6823/1028371170","heartbeat_front_addr":"172.21.15.53:6821/1028371170","state":["exists","up"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6809","nonce":2712042978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6811","nonce":2712042978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6815","nonce":2712042978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6813","nonce":2712042978}]},"public_addr":"172.21.15.53:6809/2712042978","cluster_addr":"172.21.15.53:6811/2712042978","heartbeat_back_addr":"172.21.15.53:6815/2712042978","heartbeat_front_addr":"172.21.15.53:6813/2712042978","state":["exists","up"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6801","nonce":1166517858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6803","nonce":1166517858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6807","nonce":1166517858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6805","nonce":1166517858}]},"public_addr":"172.21.15.53:6801/1166517858","cluster_addr":"172.21.15.53:6803/1166517858","heartbeat_back_addr":"172.21.15.53:6807/1166517858","heartbeat_front_addr":"172.21.15.53:6805/1166517858","state":["exists","up"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6811","nonce":1039392120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6816","nonce":1039392120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6823","nonce":1039392120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6821","nonce":1039392120}]},"public_addr":"172.21.15.144:6811/1039392120","cluster_addr":"172.21.15.144:6816/1039392120","heartbeat_back_addr":"172.21.15.144:6823/1039392120","heartbeat_front_addr":"172.21.15.144:6821/1039392120","state":["exists","up"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6810","nonce":2764095640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6813","nonce":2764095640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6820","nonce":2764095640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6817","nonce":2764095640}]},"public_addr":"172.21.15.144:6810/2764095640","cluster_addr":"172.21.15.144:6813/2764095640","heartbeat_back_addr":"172.21.15.144:6820/2764095640","heartbeat_front_addr":"172.21.15.144:6817/2764095640","state":["exists","up"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6801","nonce":1732973431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6803","nonce":1732973431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6807","nonce":1732973431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6805","nonce":1732973431}]},"public_addr":"172.21.15.144:6801/1732973431","cluster_addr":"172.21.15.144:6803/1732973431","heartbeat_back_addr":"172.21.15.144:6807/1732973431","heartbeat_front_addr":"172.21.15.144:6805/1732973431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.468749+0000","dead_epoch":31},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.044837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.310970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.241402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.877818+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.434052+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.b","osds":[0,4]},{"pgid":"3.c","osds":[1,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:18:51.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T23:18:51.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T23:18:51.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T23:18:51.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T23:18:51.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T23:18:51.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T23:18:51.842 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:51.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:18:51.851 INFO:teuthology.orchestra.run.smithi053.stdout:261993005092 2024-09-21T23:18:51.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:18:51.909 INFO:teuthology.orchestra.run.smithi053.stdout:51539607607 2024-09-21T23:18:51.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:18:51.913 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:51.913 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:18:51.923 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:51.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:18:51.931 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:51.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:18:52.246 INFO:teuthology.orchestra.run.smithi053.stdout:51539607605 2024-09-21T23:18:52.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607605 for osd.2 2024-09-21T23:18:52.278 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:52.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607607 got 51539607606 for osd.1 2024-09-21T23:18:52.401 INFO:teuthology.orchestra.run.smithi053.stdout:51539607605 2024-09-21T23:18:52.405 INFO:teuthology.orchestra.run.smithi053.stdout:51539607605 2024-09-21T23:18:52.412 INFO:teuthology.orchestra.run.smithi053.stdout:261993005090 2024-09-21T23:18:52.416 INFO:teuthology.orchestra.run.smithi053.stdout:51539607605 2024-09-21T23:18:52.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607605 for osd.3 2024-09-21T23:18:52.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607605 for osd.5 2024-09-21T23:18:52.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607605 for osd.4 2024-09-21T23:18:52.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 261993005092 got 261993005090 for osd.0 2024-09-21T23:18:53.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T23:18:53.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T23:18:53.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T23:18:53.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T23:18:53.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T23:18:53.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T23:18:53.893 INFO:teuthology.orchestra.run.smithi053.stdout:261993005092 2024-09-21T23:18:53.900 INFO:teuthology.orchestra.run.smithi053.stdout:51539607607 2024-09-21T23:18:53.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 261993005092 got 261993005092 for osd.0 2024-09-21T23:18:53.913 DEBUG:teuthology.parallel:result is None 2024-09-21T23:18:53.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607607 got 51539607607 for osd.1 2024-09-21T23:18:53.919 DEBUG:teuthology.parallel:result is None 2024-09-21T23:18:53.986 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:53.990 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:54.009 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:54.010 INFO:teuthology.orchestra.run.smithi053.stdout:51539607606 2024-09-21T23:18:54.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607606 for osd.3 2024-09-21T23:18:54.012 DEBUG:teuthology.parallel:result is None 2024-09-21T23:18:54.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607606 for osd.5 2024-09-21T23:18:54.018 DEBUG:teuthology.parallel:result is None 2024-09-21T23:18:54.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607606 for osd.2 2024-09-21T23:18:54.031 DEBUG:teuthology.parallel:result is None 2024-09-21T23:18:54.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607606 got 51539607606 for osd.4 2024-09-21T23:18:54.031 DEBUG:teuthology.parallel:result is None 2024-09-21T23:18:54.031 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T23:18:54.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:54.359 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:54.375 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-21T23:18:52.909901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":338,"num_object_clones":0,"num_object_copies":676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":330,"num_read":313971,"num_read_kb":1079449,"num_write":21392,"num_write_kb":210627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4354,"num_bytes_recovered":24942653,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20379,"ondisk_log_size":20379,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340920,"kb_used_data":15216,"kb_used_omap":13,"kb_used_meta":325554,"kb_avail":565890120,"statfs":{"total":579820584960,"available":579471482880,"internally_reserved":0,"allocated":15581184,"data_stored":7209788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13824,"internal_metadata":333367808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-550,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.129023"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13668,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790048+0000","last_change":"2024-09-21T23:18:40.802504+0000","last_active":"2024-09-21T23:18:51.790048+0000","last_peered":"2024-09-21T23:18:51.790048+0000","last_clean":"2024-09-21T23:18:51.790048+0000","last_became_active":"2024-09-21T23:18:40.801017+0000","last_became_peered":"2024-09-21T23:18:40.801017+0000","last_unstale":"2024-09-21T23:18:51.790048+0000","last_undegraded":"2024-09-21T23:18:51.790048+0000","last_fullsized":"2024-09-21T23:18:51.790048+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:02.839217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'484","reported_seq":5924,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790724+0000","last_change":"2024-09-21T23:18:46.792370+0000","last_active":"2024-09-21T23:18:51.790724+0000","last_peered":"2024-09-21T23:18:51.790724+0000","last_clean":"2024-09-21T23:18:51.790724+0000","last_became_active":"2024-09-21T23:18:46.789102+0000","last_became_peered":"2024-09-21T23:18:46.789102+0000","last_unstale":"2024-09-21T23:18:51.790724+0000","last_undegraded":"2024-09-21T23:18:51.790724+0000","last_fullsized":"2024-09-21T23:18:51.790724+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:39.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020600919999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"177'597","reported_seq":8089,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788065+0000","last_change":"2024-09-21T23:18:29.959111+0000","last_active":"2024-09-21T23:18:51.788065+0000","last_peered":"2024-09-21T23:18:51.788065+0000","last_clean":"2024-09-21T23:18:51.788065+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:51.788065+0000","last_undegraded":"2024-09-21T23:18:51.788065+0000","last_fullsized":"2024-09-21T23:18:51.788065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:19.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053022649999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"179'1015","reported_seq":13065,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787713+0000","last_change":"2024-09-21T23:18:29.959245+0000","last_active":"2024-09-21T23:18:51.787713+0000","last_peered":"2024-09-21T23:18:51.787713+0000","last_clean":"2024-09-21T23:18:51.787713+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:51.787713+0000","last_undegraded":"2024-09-21T23:18:51.787713+0000","last_fullsized":"2024-09-21T23:18:51.787713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:45.242522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048112889999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17738,"num_read_kb":71148,"num_write":1049,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'490","reported_seq":5328,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791127+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:51.791127+0000","last_peered":"2024-09-21T23:18:51.791127+0000","last_clean":"2024-09-21T23:18:51.791127+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:51.791127+0000","last_undegraded":"2024-09-21T23:18:51.791127+0000","last_fullsized":"2024-09-21T23:18:51.791127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"178'696","reported_seq":10529,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787725+0000","last_change":"2024-09-21T23:18:29.954505+0000","last_active":"2024-09-21T23:18:51.787725+0000","last_peered":"2024-09-21T23:18:51.787725+0000","last_clean":"2024-09-21T23:18:51.787725+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:51.787725+0000","last_undegraded":"2024-09-21T23:18:51.787725+0000","last_fullsized":"2024-09-21T23:18:51.787725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:47.554016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059155759999999996,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12157,"num_read_kb":26818,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'411","reported_seq":5443,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029539+0000","last_change":"2024-09-21T23:18:30.016900+0000","last_active":"2024-09-21T23:18:51.029539+0000","last_peered":"2024-09-21T23:18:51.029539+0000","last_clean":"2024-09-21T23:18:51.029539+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:51.029539+0000","last_undegraded":"2024-09-21T23:18:51.029539+0000","last_fullsized":"2024-09-21T23:18:51.029539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:28.556199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016355880000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'672","reported_seq":18660,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790494+0000","last_change":"2024-09-21T23:18:47.800021+0000","last_active":"2024-09-21T23:18:51.790494+0000","last_peered":"2024-09-21T23:18:51.790494+0000","last_clean":"2024-09-21T23:18:51.790494+0000","last_became_active":"2024-09-21T23:18:47.797088+0000","last_became_peered":"2024-09-21T23:18:47.797088+0000","last_unstale":"2024-09-21T23:18:51.790494+0000","last_undegraded":"2024-09-21T23:18:51.790494+0000","last_fullsized":"2024-09-21T23:18:51.790494+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:31.792480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"178'416","reported_seq":13674,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790017+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:51.790017+0000","last_peered":"2024-09-21T23:18:51.790017+0000","last_clean":"2024-09-21T23:18:51.790017+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:51.790017+0000","last_undegraded":"2024-09-21T23:18:51.790017+0000","last_fullsized":"2024-09-21T23:18:51.790017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"178'581","reported_seq":6245,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788309+0000","last_change":"2024-09-21T23:18:39.802796+0000","last_active":"2024-09-21T23:18:51.788309+0000","last_peered":"2024-09-21T23:18:51.788309+0000","last_clean":"2024-09-21T23:18:51.788309+0000","last_became_active":"2024-09-21T23:18:39.798603+0000","last_became_peered":"2024-09-21T23:18:39.798603+0000","last_unstale":"2024-09-21T23:18:51.788309+0000","last_undegraded":"2024-09-21T23:18:51.788309+0000","last_fullsized":"2024-09-21T23:18:51.788309+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:00:20.739193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061281979999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"177'349","reported_seq":4291,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028231+0000","last_change":"2024-09-21T23:18:38.796650+0000","last_active":"2024-09-21T23:18:51.028231+0000","last_peered":"2024-09-21T23:18:51.028231+0000","last_clean":"2024-09-21T23:18:51.028231+0000","last_became_active":"2024-09-21T23:18:38.791983+0000","last_became_peered":"2024-09-21T23:18:38.791983+0000","last_unstale":"2024-09-21T23:18:51.028231+0000","last_undegraded":"2024-09-21T23:18:51.028231+0000","last_fullsized":"2024-09-21T23:18:51.028231+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:55.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"178'938","reported_seq":8298,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028293+0000","last_change":"2024-09-21T23:18:39.806128+0000","last_active":"2024-09-21T23:18:51.028293+0000","last_peered":"2024-09-21T23:18:51.028293+0000","last_clean":"2024-09-21T23:18:51.028293+0000","last_became_active":"2024-09-21T23:18:39.801495+0000","last_became_peered":"2024-09-21T23:18:39.801495+0000","last_unstale":"2024-09-21T23:18:51.028293+0000","last_undegraded":"2024-09-21T23:18:51.028293+0000","last_fullsized":"2024-09-21T23:18:51.028293+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:08.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074401700000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028633+0000","last_change":"2024-09-21T23:18:33.987293+0000","last_active":"2024-09-21T23:18:51.028633+0000","last_peered":"2024-09-21T23:18:51.028633+0000","last_clean":"2024-09-21T23:18:51.028633+0000","last_became_active":"2024-09-21T23:18:33.986262+0000","last_became_peered":"2024-09-21T23:18:33.986262+0000","last_unstale":"2024-09-21T23:18:51.028633+0000","last_undegraded":"2024-09-21T23:18:51.028633+0000","last_fullsized":"2024-09-21T23:18:51.028633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:53.298589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20288,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790013+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:51.790013+0000","last_peered":"2024-09-21T23:18:51.790013+0000","last_clean":"2024-09-21T23:18:51.790013+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:51.790013+0000","last_undegraded":"2024-09-21T23:18:51.790013+0000","last_fullsized":"2024-09-21T23:18:51.790013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029619+0000","last_change":"2024-09-21T23:18:33.986624+0000","last_active":"2024-09-21T23:18:51.029619+0000","last_peered":"2024-09-21T23:18:51.029619+0000","last_clean":"2024-09-21T23:18:51.029619+0000","last_became_active":"2024-09-21T23:18:33.986227+0000","last_became_peered":"2024-09-21T23:18:33.986227+0000","last_unstale":"2024-09-21T23:18:51.029619+0000","last_undegraded":"2024-09-21T23:18:51.029619+0000","last_fullsized":"2024-09-21T23:18:51.029619+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:46.791389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8599,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790716+0000","last_change":"2024-09-21T23:18:48.797459+0000","last_active":"2024-09-21T23:18:51.790716+0000","last_peered":"2024-09-21T23:18:51.790716+0000","last_clean":"2024-09-21T23:18:51.790716+0000","last_became_active":"2024-09-21T23:18:48.795281+0000","last_became_peered":"2024-09-21T23:18:48.795281+0000","last_unstale":"2024-09-21T23:18:51.790716+0000","last_undegraded":"2024-09-21T23:18:51.790716+0000","last_fullsized":"2024-09-21T23:18:51.790716+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:12:45.760380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244768,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028468+0000","last_change":"2024-09-21T23:18:33.982766+0000","last_active":"2024-09-21T23:18:51.028468+0000","last_peered":"2024-09-21T23:18:51.028468+0000","last_clean":"2024-09-21T23:18:51.028468+0000","last_became_active":"2024-09-21T23:18:33.981926+0000","last_became_peered":"2024-09-21T23:18:33.981926+0000","last_unstale":"2024-09-21T23:18:51.028468+0000","last_undegraded":"2024-09-21T23:18:51.028468+0000","last_fullsized":"2024-09-21T23:18:51.028468+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:06.668878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":3308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029990+0000","last_change":"2024-09-21T23:18:47.796589+0000","last_active":"2024-09-21T23:18:51.029990+0000","last_peered":"2024-09-21T23:18:51.029990+0000","last_clean":"2024-09-21T23:18:51.029990+0000","last_became_active":"2024-09-21T23:18:47.794405+0000","last_became_peered":"2024-09-21T23:18:47.794405+0000","last_unstale":"2024-09-21T23:18:51.029990+0000","last_undegraded":"2024-09-21T23:18:51.029990+0000","last_fullsized":"2024-09-21T23:18:51.029990+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:07:32.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073231999999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":269,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029959+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:51.029959+0000","last_peered":"2024-09-21T23:18:51.029959+0000","last_clean":"2024-09-21T23:18:51.029959+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:51.029959+0000","last_undegraded":"2024-09-21T23:18:51.029959+0000","last_fullsized":"2024-09-21T23:18:51.029959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":11175,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788244+0000","last_change":"2024-09-21T23:18:42.790134+0000","last_active":"2024-09-21T23:18:51.788244+0000","last_peered":"2024-09-21T23:18:51.788244+0000","last_clean":"2024-09-21T23:18:51.788244+0000","last_became_active":"2024-09-21T23:18:42.787666+0000","last_became_peered":"2024-09-21T23:18:42.787666+0000","last_unstale":"2024-09-21T23:18:51.788244+0000","last_undegraded":"2024-09-21T23:18:51.788244+0000","last_fullsized":"2024-09-21T23:18:51.788244+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:20.552688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069917950000000003,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'792","reported_seq":7120,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787876+0000","last_change":"2024-09-21T23:18:29.953189+0000","last_active":"2024-09-21T23:18:51.787876+0000","last_peered":"2024-09-21T23:18:51.787876+0000","last_clean":"2024-09-21T23:18:51.787876+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:51.787876+0000","last_undegraded":"2024-09-21T23:18:51.787876+0000","last_fullsized":"2024-09-21T23:18:51.787876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:20.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046990039999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8412,"num_read_kb":29133,"num_write":771,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":404,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787751+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:51.787751+0000","last_peered":"2024-09-21T23:18:51.787751+0000","last_clean":"2024-09-21T23:18:51.787751+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:51.787751+0000","last_undegraded":"2024-09-21T23:18:51.787751+0000","last_fullsized":"2024-09-21T23:18:51.787751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6682,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029845+0000","last_change":"2024-09-21T23:18:47.800836+0000","last_active":"2024-09-21T23:18:51.029845+0000","last_peered":"2024-09-21T23:18:51.029845+0000","last_clean":"2024-09-21T23:18:51.029845+0000","last_became_active":"2024-09-21T23:18:47.798894+0000","last_became_peered":"2024-09-21T23:18:47.798894+0000","last_unstale":"2024-09-21T23:18:51.029845+0000","last_undegraded":"2024-09-21T23:18:51.029845+0000","last_fullsized":"2024-09-21T23:18:51.029845+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:19.199753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015079399999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790577+0000","last_change":"2024-09-21T23:18:42.788361+0000","last_active":"2024-09-21T23:18:51.790577+0000","last_peered":"2024-09-21T23:18:51.790577+0000","last_clean":"2024-09-21T23:18:51.790577+0000","last_became_active":"2024-09-21T23:18:42.788015+0000","last_became_peered":"2024-09-21T23:18:42.788015+0000","last_unstale":"2024-09-21T23:18:51.790577+0000","last_undegraded":"2024-09-21T23:18:51.790577+0000","last_fullsized":"2024-09-21T23:18:51.790577+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:44.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4792,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790731+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:51.790731+0000","last_peered":"2024-09-21T23:18:51.790731+0000","last_clean":"2024-09-21T23:18:51.790731+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:51.790731+0000","last_undegraded":"2024-09-21T23:18:51.790731+0000","last_fullsized":"2024-09-21T23:18:51.790731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027766+0000","last_change":"2024-09-21T23:18:33.978285+0000","last_active":"2024-09-21T23:18:51.027766+0000","last_peered":"2024-09-21T23:18:51.027766+0000","last_clean":"2024-09-21T23:18:51.027766+0000","last_became_active":"2024-09-21T23:18:33.977449+0000","last_became_peered":"2024-09-21T23:18:33.977449+0000","last_unstale":"2024-09-21T23:18:51.027766+0000","last_undegraded":"2024-09-21T23:18:51.027766+0000","last_fullsized":"2024-09-21T23:18:51.027766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:03.716381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":17031,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029958+0000","last_change":"2024-09-21T23:18:30.017566+0000","last_active":"2024-09-21T23:18:51.029958+0000","last_peered":"2024-09-21T23:18:51.029958+0000","last_clean":"2024-09-21T23:18:51.029958+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:51.029958+0000","last_undegraded":"2024-09-21T23:18:51.029958+0000","last_fullsized":"2024-09-21T23:18:51.029958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:56.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020263540000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788003+0000","last_change":"2024-09-21T23:18:42.793227+0000","last_active":"2024-09-21T23:18:51.788003+0000","last_peered":"2024-09-21T23:18:51.788003+0000","last_clean":"2024-09-21T23:18:51.788003+0000","last_became_active":"2024-09-21T23:18:42.788526+0000","last_became_peered":"2024-09-21T23:18:42.788526+0000","last_unstale":"2024-09-21T23:18:51.788003+0000","last_undegraded":"2024-09-21T23:18:51.788003+0000","last_fullsized":"2024-09-21T23:18:51.788003+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:53.455486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":483,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027862+0000","last_change":"2024-09-21T23:18:36.778564+0000","last_active":"2024-09-21T23:18:51.027862+0000","last_peered":"2024-09-21T23:18:51.027862+0000","last_clean":"2024-09-21T23:18:51.027862+0000","last_became_active":"2024-09-21T23:18:36.777596+0000","last_became_peered":"2024-09-21T23:18:36.777596+0000","last_unstale":"2024-09-21T23:18:51.027862+0000","last_undegraded":"2024-09-21T23:18:51.027862+0000","last_fullsized":"2024-09-21T23:18:51.027862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:32.921245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12928,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790372+0000","last_change":"2024-09-21T23:18:47.797129+0000","last_active":"2024-09-21T23:18:51.790372+0000","last_peered":"2024-09-21T23:18:51.790372+0000","last_clean":"2024-09-21T23:18:51.790372+0000","last_became_active":"2024-09-21T23:18:47.793978+0000","last_became_peered":"2024-09-21T23:18:47.793978+0000","last_unstale":"2024-09-21T23:18:51.790372+0000","last_undegraded":"2024-09-21T23:18:51.790372+0000","last_fullsized":"2024-09-21T23:18:51.790372+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:42.993883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011535569999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"178'592","reported_seq":5301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787636+0000","last_change":"2024-09-21T23:18:46.790171+0000","last_active":"2024-09-21T23:18:51.787636+0000","last_peered":"2024-09-21T23:18:51.787636+0000","last_clean":"2024-09-21T23:18:51.787636+0000","last_became_active":"2024-09-21T23:18:46.787797+0000","last_became_peered":"2024-09-21T23:18:46.787797+0000","last_unstale":"2024-09-21T23:18:51.787636+0000","last_undegraded":"2024-09-21T23:18:51.787636+0000","last_fullsized":"2024-09-21T23:18:51.787636+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:43:38.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085867570000000004,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"178'354","reported_seq":4260,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027978+0000","last_change":"2024-09-21T23:18:38.801696+0000","last_active":"2024-09-21T23:18:51.027978+0000","last_peered":"2024-09-21T23:18:51.027978+0000","last_clean":"2024-09-21T23:18:51.027978+0000","last_became_active":"2024-09-21T23:18:38.795761+0000","last_became_peered":"2024-09-21T23:18:38.795761+0000","last_unstale":"2024-09-21T23:18:51.027978+0000","last_undegraded":"2024-09-21T23:18:51.027978+0000","last_fullsized":"2024-09-21T23:18:51.027978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:33.754942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"177'791","reported_seq":8669,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791035+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:51.791035+0000","last_peered":"2024-09-21T23:18:51.791035+0000","last_clean":"2024-09-21T23:18:51.791035+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:51.791035+0000","last_undegraded":"2024-09-21T23:18:51.791035+0000","last_fullsized":"2024-09-21T23:18:51.791035+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"179'686","reported_seq":6156,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029573+0000","last_change":"2024-09-21T23:18:42.790726+0000","last_active":"2024-09-21T23:18:51.029573+0000","last_peered":"2024-09-21T23:18:51.029573+0000","last_clean":"2024-09-21T23:18:51.029573+0000","last_became_active":"2024-09-21T23:18:42.788002+0000","last_became_peered":"2024-09-21T23:18:42.788002+0000","last_unstale":"2024-09-21T23:18:51.029573+0000","last_undegraded":"2024-09-21T23:18:51.029573+0000","last_fullsized":"2024-09-21T23:18:51.029573+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:04:27.488128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062598719999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7976,"num_read_kb":18238,"num_write":516,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"177'475","reported_seq":5551,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028181+0000","last_change":"2024-09-21T23:18:46.790120+0000","last_active":"2024-09-21T23:18:51.028181+0000","last_peered":"2024-09-21T23:18:51.028181+0000","last_clean":"2024-09-21T23:18:51.028181+0000","last_became_active":"2024-09-21T23:18:46.786927+0000","last_became_peered":"2024-09-21T23:18:46.786927+0000","last_unstale":"2024-09-21T23:18:51.028181+0000","last_undegraded":"2024-09-21T23:18:51.028181+0000","last_fullsized":"2024-09-21T23:18:51.028181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:24:25.703682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003514428,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"173'472","reported_seq":4619,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788417+0000","last_change":"2024-09-21T23:18:29.962695+0000","last_active":"2024-09-21T23:18:51.788417+0000","last_peered":"2024-09-21T23:18:51.788417+0000","last_clean":"2024-09-21T23:18:51.788417+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:51.788417+0000","last_undegraded":"2024-09-21T23:18:51.788417+0000","last_fullsized":"2024-09-21T23:18:51.788417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:37.106862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073184640000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'479","reported_seq":6832,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788736+0000","last_change":"2024-09-21T23:18:36.787585+0000","last_active":"2024-09-21T23:18:51.788736+0000","last_peered":"2024-09-21T23:18:51.788736+0000","last_clean":"2024-09-21T23:18:51.788736+0000","last_became_active":"2024-09-21T23:18:36.784584+0000","last_became_peered":"2024-09-21T23:18:36.784584+0000","last_unstale":"2024-09-21T23:18:51.788736+0000","last_undegraded":"2024-09-21T23:18:51.788736+0000","last_fullsized":"2024-09-21T23:18:51.788736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:10:34.977998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080862480000000007,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"178'776","reported_seq":6199,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788078+0000","last_change":"2024-09-21T23:18:29.956286+0000","last_active":"2024-09-21T23:18:51.788078+0000","last_peered":"2024-09-21T23:18:51.788078+0000","last_clean":"2024-09-21T23:18:51.788078+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:51.788078+0000","last_undegraded":"2024-09-21T23:18:51.788078+0000","last_fullsized":"2024-09-21T23:18:51.788078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:23:05.964943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028297349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"178'539","reported_seq":5572,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027938+0000","last_change":"2024-09-21T23:18:40.806974+0000","last_active":"2024-09-21T23:18:51.027938+0000","last_peered":"2024-09-21T23:18:51.027938+0000","last_clean":"2024-09-21T23:18:51.027938+0000","last_became_active":"2024-09-21T23:18:40.802654+0000","last_became_peered":"2024-09-21T23:18:40.802654+0000","last_unstale":"2024-09-21T23:18:51.027938+0000","last_undegraded":"2024-09-21T23:18:51.027938+0000","last_fullsized":"2024-09-21T23:18:51.027938+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:51.646949+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029604689999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"177'807","reported_seq":22299,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790855+0000","last_change":"2024-09-21T23:18:40.819974+0000","last_active":"2024-09-21T23:18:51.790855+0000","last_peered":"2024-09-21T23:18:51.790855+0000","last_clean":"2024-09-21T23:18:51.790855+0000","last_became_active":"2024-09-21T23:18:40.816843+0000","last_became_peered":"2024-09-21T23:18:40.816843+0000","last_unstale":"2024-09-21T23:18:51.790855+0000","last_undegraded":"2024-09-21T23:18:51.790855+0000","last_fullsized":"2024-09-21T23:18:51.790855+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:12.452126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"179'484","reported_seq":9830,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028099+0000","last_change":"2024-09-21T23:18:38.798246+0000","last_active":"2024-09-21T23:18:51.028099+0000","last_peered":"2024-09-21T23:18:51.028099+0000","last_clean":"2024-09-21T23:18:51.028099+0000","last_became_active":"2024-09-21T23:18:38.791632+0000","last_became_peered":"2024-09-21T23:18:38.791632+0000","last_unstale":"2024-09-21T23:18:51.028099+0000","last_undegraded":"2024-09-21T23:18:51.028099+0000","last_fullsized":"2024-09-21T23:18:51.028099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:30.701459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025997469999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":527,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":330,"num_read":313855,"num_read_kb":1079349,"num_write":21231,"num_write_kb":208209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4344,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20259,"ondisk_log_size":20259,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607606,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51580,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320260,"statfs":{"total":96636764160,"available":96583946240,"internally_reserved":0,"allocated":2396160,"data_stored":1003182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55736,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316104,"statfs":{"total":96636764160,"available":96579690496,"internally_reserved":0,"allocated":2396160,"data_stored":1003182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607606,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42436,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329404,"statfs":{"total":96636764160,"available":96593309696,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607606,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2392064,"data_stored":1003163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77968,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293872,"statfs":{"total":96636764160,"available":96556924928,"internally_reserved":0,"allocated":2412544,"data_stored":1008200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005092,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":2916,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":2985984,"data_stored":1593531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:54.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:54.674 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:54.674 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:54.690 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-21T23:18:52.909901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":338,"num_object_clones":0,"num_object_copies":676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":330,"num_read":313971,"num_read_kb":1079449,"num_write":21392,"num_write_kb":210627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4354,"num_bytes_recovered":24942653,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20379,"ondisk_log_size":20379,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340920,"kb_used_data":15216,"kb_used_omap":13,"kb_used_meta":325554,"kb_avail":565890120,"statfs":{"total":579820584960,"available":579471482880,"internally_reserved":0,"allocated":15581184,"data_stored":7209788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13824,"internal_metadata":333367808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-550,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.129023"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13668,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790048+0000","last_change":"2024-09-21T23:18:40.802504+0000","last_active":"2024-09-21T23:18:51.790048+0000","last_peered":"2024-09-21T23:18:51.790048+0000","last_clean":"2024-09-21T23:18:51.790048+0000","last_became_active":"2024-09-21T23:18:40.801017+0000","last_became_peered":"2024-09-21T23:18:40.801017+0000","last_unstale":"2024-09-21T23:18:51.790048+0000","last_undegraded":"2024-09-21T23:18:51.790048+0000","last_fullsized":"2024-09-21T23:18:51.790048+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:02.839217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'484","reported_seq":5924,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790724+0000","last_change":"2024-09-21T23:18:46.792370+0000","last_active":"2024-09-21T23:18:51.790724+0000","last_peered":"2024-09-21T23:18:51.790724+0000","last_clean":"2024-09-21T23:18:51.790724+0000","last_became_active":"2024-09-21T23:18:46.789102+0000","last_became_peered":"2024-09-21T23:18:46.789102+0000","last_unstale":"2024-09-21T23:18:51.790724+0000","last_undegraded":"2024-09-21T23:18:51.790724+0000","last_fullsized":"2024-09-21T23:18:51.790724+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:39.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020600919999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"177'597","reported_seq":8089,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788065+0000","last_change":"2024-09-21T23:18:29.959111+0000","last_active":"2024-09-21T23:18:51.788065+0000","last_peered":"2024-09-21T23:18:51.788065+0000","last_clean":"2024-09-21T23:18:51.788065+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:51.788065+0000","last_undegraded":"2024-09-21T23:18:51.788065+0000","last_fullsized":"2024-09-21T23:18:51.788065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:19.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053022649999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"179'1015","reported_seq":13065,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787713+0000","last_change":"2024-09-21T23:18:29.959245+0000","last_active":"2024-09-21T23:18:51.787713+0000","last_peered":"2024-09-21T23:18:51.787713+0000","last_clean":"2024-09-21T23:18:51.787713+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:51.787713+0000","last_undegraded":"2024-09-21T23:18:51.787713+0000","last_fullsized":"2024-09-21T23:18:51.787713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:45.242522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048112889999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17738,"num_read_kb":71148,"num_write":1049,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'490","reported_seq":5328,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791127+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:51.791127+0000","last_peered":"2024-09-21T23:18:51.791127+0000","last_clean":"2024-09-21T23:18:51.791127+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:51.791127+0000","last_undegraded":"2024-09-21T23:18:51.791127+0000","last_fullsized":"2024-09-21T23:18:51.791127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"178'696","reported_seq":10529,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787725+0000","last_change":"2024-09-21T23:18:29.954505+0000","last_active":"2024-09-21T23:18:51.787725+0000","last_peered":"2024-09-21T23:18:51.787725+0000","last_clean":"2024-09-21T23:18:51.787725+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:51.787725+0000","last_undegraded":"2024-09-21T23:18:51.787725+0000","last_fullsized":"2024-09-21T23:18:51.787725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:47.554016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059155759999999996,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12157,"num_read_kb":26818,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'411","reported_seq":5443,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029539+0000","last_change":"2024-09-21T23:18:30.016900+0000","last_active":"2024-09-21T23:18:51.029539+0000","last_peered":"2024-09-21T23:18:51.029539+0000","last_clean":"2024-09-21T23:18:51.029539+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:51.029539+0000","last_undegraded":"2024-09-21T23:18:51.029539+0000","last_fullsized":"2024-09-21T23:18:51.029539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:28.556199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016355880000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'672","reported_seq":18660,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790494+0000","last_change":"2024-09-21T23:18:47.800021+0000","last_active":"2024-09-21T23:18:51.790494+0000","last_peered":"2024-09-21T23:18:51.790494+0000","last_clean":"2024-09-21T23:18:51.790494+0000","last_became_active":"2024-09-21T23:18:47.797088+0000","last_became_peered":"2024-09-21T23:18:47.797088+0000","last_unstale":"2024-09-21T23:18:51.790494+0000","last_undegraded":"2024-09-21T23:18:51.790494+0000","last_fullsized":"2024-09-21T23:18:51.790494+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:31.792480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"178'416","reported_seq":13674,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790017+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:51.790017+0000","last_peered":"2024-09-21T23:18:51.790017+0000","last_clean":"2024-09-21T23:18:51.790017+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:51.790017+0000","last_undegraded":"2024-09-21T23:18:51.790017+0000","last_fullsized":"2024-09-21T23:18:51.790017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"178'581","reported_seq":6245,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788309+0000","last_change":"2024-09-21T23:18:39.802796+0000","last_active":"2024-09-21T23:18:51.788309+0000","last_peered":"2024-09-21T23:18:51.788309+0000","last_clean":"2024-09-21T23:18:51.788309+0000","last_became_active":"2024-09-21T23:18:39.798603+0000","last_became_peered":"2024-09-21T23:18:39.798603+0000","last_unstale":"2024-09-21T23:18:51.788309+0000","last_undegraded":"2024-09-21T23:18:51.788309+0000","last_fullsized":"2024-09-21T23:18:51.788309+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:00:20.739193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061281979999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"177'349","reported_seq":4291,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028231+0000","last_change":"2024-09-21T23:18:38.796650+0000","last_active":"2024-09-21T23:18:51.028231+0000","last_peered":"2024-09-21T23:18:51.028231+0000","last_clean":"2024-09-21T23:18:51.028231+0000","last_became_active":"2024-09-21T23:18:38.791983+0000","last_became_peered":"2024-09-21T23:18:38.791983+0000","last_unstale":"2024-09-21T23:18:51.028231+0000","last_undegraded":"2024-09-21T23:18:51.028231+0000","last_fullsized":"2024-09-21T23:18:51.028231+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:55.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"178'938","reported_seq":8298,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028293+0000","last_change":"2024-09-21T23:18:39.806128+0000","last_active":"2024-09-21T23:18:51.028293+0000","last_peered":"2024-09-21T23:18:51.028293+0000","last_clean":"2024-09-21T23:18:51.028293+0000","last_became_active":"2024-09-21T23:18:39.801495+0000","last_became_peered":"2024-09-21T23:18:39.801495+0000","last_unstale":"2024-09-21T23:18:51.028293+0000","last_undegraded":"2024-09-21T23:18:51.028293+0000","last_fullsized":"2024-09-21T23:18:51.028293+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:08.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074401700000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028633+0000","last_change":"2024-09-21T23:18:33.987293+0000","last_active":"2024-09-21T23:18:51.028633+0000","last_peered":"2024-09-21T23:18:51.028633+0000","last_clean":"2024-09-21T23:18:51.028633+0000","last_became_active":"2024-09-21T23:18:33.986262+0000","last_became_peered":"2024-09-21T23:18:33.986262+0000","last_unstale":"2024-09-21T23:18:51.028633+0000","last_undegraded":"2024-09-21T23:18:51.028633+0000","last_fullsized":"2024-09-21T23:18:51.028633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:53.298589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20288,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790013+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:51.790013+0000","last_peered":"2024-09-21T23:18:51.790013+0000","last_clean":"2024-09-21T23:18:51.790013+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:51.790013+0000","last_undegraded":"2024-09-21T23:18:51.790013+0000","last_fullsized":"2024-09-21T23:18:51.790013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029619+0000","last_change":"2024-09-21T23:18:33.986624+0000","last_active":"2024-09-21T23:18:51.029619+0000","last_peered":"2024-09-21T23:18:51.029619+0000","last_clean":"2024-09-21T23:18:51.029619+0000","last_became_active":"2024-09-21T23:18:33.986227+0000","last_became_peered":"2024-09-21T23:18:33.986227+0000","last_unstale":"2024-09-21T23:18:51.029619+0000","last_undegraded":"2024-09-21T23:18:51.029619+0000","last_fullsized":"2024-09-21T23:18:51.029619+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:46.791389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8599,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790716+0000","last_change":"2024-09-21T23:18:48.797459+0000","last_active":"2024-09-21T23:18:51.790716+0000","last_peered":"2024-09-21T23:18:51.790716+0000","last_clean":"2024-09-21T23:18:51.790716+0000","last_became_active":"2024-09-21T23:18:48.795281+0000","last_became_peered":"2024-09-21T23:18:48.795281+0000","last_unstale":"2024-09-21T23:18:51.790716+0000","last_undegraded":"2024-09-21T23:18:51.790716+0000","last_fullsized":"2024-09-21T23:18:51.790716+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:12:45.760380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244768,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028468+0000","last_change":"2024-09-21T23:18:33.982766+0000","last_active":"2024-09-21T23:18:51.028468+0000","last_peered":"2024-09-21T23:18:51.028468+0000","last_clean":"2024-09-21T23:18:51.028468+0000","last_became_active":"2024-09-21T23:18:33.981926+0000","last_became_peered":"2024-09-21T23:18:33.981926+0000","last_unstale":"2024-09-21T23:18:51.028468+0000","last_undegraded":"2024-09-21T23:18:51.028468+0000","last_fullsized":"2024-09-21T23:18:51.028468+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:06.668878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":3308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029990+0000","last_change":"2024-09-21T23:18:47.796589+0000","last_active":"2024-09-21T23:18:51.029990+0000","last_peered":"2024-09-21T23:18:51.029990+0000","last_clean":"2024-09-21T23:18:51.029990+0000","last_became_active":"2024-09-21T23:18:47.794405+0000","last_became_peered":"2024-09-21T23:18:47.794405+0000","last_unstale":"2024-09-21T23:18:51.029990+0000","last_undegraded":"2024-09-21T23:18:51.029990+0000","last_fullsized":"2024-09-21T23:18:51.029990+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:07:32.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073231999999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":269,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029959+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:51.029959+0000","last_peered":"2024-09-21T23:18:51.029959+0000","last_clean":"2024-09-21T23:18:51.029959+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:51.029959+0000","last_undegraded":"2024-09-21T23:18:51.029959+0000","last_fullsized":"2024-09-21T23:18:51.029959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":11175,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788244+0000","last_change":"2024-09-21T23:18:42.790134+0000","last_active":"2024-09-21T23:18:51.788244+0000","last_peered":"2024-09-21T23:18:51.788244+0000","last_clean":"2024-09-21T23:18:51.788244+0000","last_became_active":"2024-09-21T23:18:42.787666+0000","last_became_peered":"2024-09-21T23:18:42.787666+0000","last_unstale":"2024-09-21T23:18:51.788244+0000","last_undegraded":"2024-09-21T23:18:51.788244+0000","last_fullsized":"2024-09-21T23:18:51.788244+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:20.552688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069917950000000003,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'792","reported_seq":7120,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787876+0000","last_change":"2024-09-21T23:18:29.953189+0000","last_active":"2024-09-21T23:18:51.787876+0000","last_peered":"2024-09-21T23:18:51.787876+0000","last_clean":"2024-09-21T23:18:51.787876+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:51.787876+0000","last_undegraded":"2024-09-21T23:18:51.787876+0000","last_fullsized":"2024-09-21T23:18:51.787876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:20.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046990039999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8412,"num_read_kb":29133,"num_write":771,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":404,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787751+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:51.787751+0000","last_peered":"2024-09-21T23:18:51.787751+0000","last_clean":"2024-09-21T23:18:51.787751+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:51.787751+0000","last_undegraded":"2024-09-21T23:18:51.787751+0000","last_fullsized":"2024-09-21T23:18:51.787751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6682,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029845+0000","last_change":"2024-09-21T23:18:47.800836+0000","last_active":"2024-09-21T23:18:51.029845+0000","last_peered":"2024-09-21T23:18:51.029845+0000","last_clean":"2024-09-21T23:18:51.029845+0000","last_became_active":"2024-09-21T23:18:47.798894+0000","last_became_peered":"2024-09-21T23:18:47.798894+0000","last_unstale":"2024-09-21T23:18:51.029845+0000","last_undegraded":"2024-09-21T23:18:51.029845+0000","last_fullsized":"2024-09-21T23:18:51.029845+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:19.199753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015079399999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790577+0000","last_change":"2024-09-21T23:18:42.788361+0000","last_active":"2024-09-21T23:18:51.790577+0000","last_peered":"2024-09-21T23:18:51.790577+0000","last_clean":"2024-09-21T23:18:51.790577+0000","last_became_active":"2024-09-21T23:18:42.788015+0000","last_became_peered":"2024-09-21T23:18:42.788015+0000","last_unstale":"2024-09-21T23:18:51.790577+0000","last_undegraded":"2024-09-21T23:18:51.790577+0000","last_fullsized":"2024-09-21T23:18:51.790577+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:44.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4792,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790731+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:51.790731+0000","last_peered":"2024-09-21T23:18:51.790731+0000","last_clean":"2024-09-21T23:18:51.790731+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:51.790731+0000","last_undegraded":"2024-09-21T23:18:51.790731+0000","last_fullsized":"2024-09-21T23:18:51.790731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027766+0000","last_change":"2024-09-21T23:18:33.978285+0000","last_active":"2024-09-21T23:18:51.027766+0000","last_peered":"2024-09-21T23:18:51.027766+0000","last_clean":"2024-09-21T23:18:51.027766+0000","last_became_active":"2024-09-21T23:18:33.977449+0000","last_became_peered":"2024-09-21T23:18:33.977449+0000","last_unstale":"2024-09-21T23:18:51.027766+0000","last_undegraded":"2024-09-21T23:18:51.027766+0000","last_fullsized":"2024-09-21T23:18:51.027766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:03.716381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":17031,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029958+0000","last_change":"2024-09-21T23:18:30.017566+0000","last_active":"2024-09-21T23:18:51.029958+0000","last_peered":"2024-09-21T23:18:51.029958+0000","last_clean":"2024-09-21T23:18:51.029958+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:51.029958+0000","last_undegraded":"2024-09-21T23:18:51.029958+0000","last_fullsized":"2024-09-21T23:18:51.029958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:56.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020263540000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788003+0000","last_change":"2024-09-21T23:18:42.793227+0000","last_active":"2024-09-21T23:18:51.788003+0000","last_peered":"2024-09-21T23:18:51.788003+0000","last_clean":"2024-09-21T23:18:51.788003+0000","last_became_active":"2024-09-21T23:18:42.788526+0000","last_became_peered":"2024-09-21T23:18:42.788526+0000","last_unstale":"2024-09-21T23:18:51.788003+0000","last_undegraded":"2024-09-21T23:18:51.788003+0000","last_fullsized":"2024-09-21T23:18:51.788003+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:53.455486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":483,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027862+0000","last_change":"2024-09-21T23:18:36.778564+0000","last_active":"2024-09-21T23:18:51.027862+0000","last_peered":"2024-09-21T23:18:51.027862+0000","last_clean":"2024-09-21T23:18:51.027862+0000","last_became_active":"2024-09-21T23:18:36.777596+0000","last_became_peered":"2024-09-21T23:18:36.777596+0000","last_unstale":"2024-09-21T23:18:51.027862+0000","last_undegraded":"2024-09-21T23:18:51.027862+0000","last_fullsized":"2024-09-21T23:18:51.027862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:32.921245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12928,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790372+0000","last_change":"2024-09-21T23:18:47.797129+0000","last_active":"2024-09-21T23:18:51.790372+0000","last_peered":"2024-09-21T23:18:51.790372+0000","last_clean":"2024-09-21T23:18:51.790372+0000","last_became_active":"2024-09-21T23:18:47.793978+0000","last_became_peered":"2024-09-21T23:18:47.793978+0000","last_unstale":"2024-09-21T23:18:51.790372+0000","last_undegraded":"2024-09-21T23:18:51.790372+0000","last_fullsized":"2024-09-21T23:18:51.790372+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:42.993883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011535569999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"178'592","reported_seq":5301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787636+0000","last_change":"2024-09-21T23:18:46.790171+0000","last_active":"2024-09-21T23:18:51.787636+0000","last_peered":"2024-09-21T23:18:51.787636+0000","last_clean":"2024-09-21T23:18:51.787636+0000","last_became_active":"2024-09-21T23:18:46.787797+0000","last_became_peered":"2024-09-21T23:18:46.787797+0000","last_unstale":"2024-09-21T23:18:51.787636+0000","last_undegraded":"2024-09-21T23:18:51.787636+0000","last_fullsized":"2024-09-21T23:18:51.787636+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:43:38.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085867570000000004,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"178'354","reported_seq":4260,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027978+0000","last_change":"2024-09-21T23:18:38.801696+0000","last_active":"2024-09-21T23:18:51.027978+0000","last_peered":"2024-09-21T23:18:51.027978+0000","last_clean":"2024-09-21T23:18:51.027978+0000","last_became_active":"2024-09-21T23:18:38.795761+0000","last_became_peered":"2024-09-21T23:18:38.795761+0000","last_unstale":"2024-09-21T23:18:51.027978+0000","last_undegraded":"2024-09-21T23:18:51.027978+0000","last_fullsized":"2024-09-21T23:18:51.027978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:33.754942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"177'791","reported_seq":8669,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791035+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:51.791035+0000","last_peered":"2024-09-21T23:18:51.791035+0000","last_clean":"2024-09-21T23:18:51.791035+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:51.791035+0000","last_undegraded":"2024-09-21T23:18:51.791035+0000","last_fullsized":"2024-09-21T23:18:51.791035+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"179'686","reported_seq":6156,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029573+0000","last_change":"2024-09-21T23:18:42.790726+0000","last_active":"2024-09-21T23:18:51.029573+0000","last_peered":"2024-09-21T23:18:51.029573+0000","last_clean":"2024-09-21T23:18:51.029573+0000","last_became_active":"2024-09-21T23:18:42.788002+0000","last_became_peered":"2024-09-21T23:18:42.788002+0000","last_unstale":"2024-09-21T23:18:51.029573+0000","last_undegraded":"2024-09-21T23:18:51.029573+0000","last_fullsized":"2024-09-21T23:18:51.029573+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:04:27.488128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062598719999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7976,"num_read_kb":18238,"num_write":516,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"177'475","reported_seq":5551,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028181+0000","last_change":"2024-09-21T23:18:46.790120+0000","last_active":"2024-09-21T23:18:51.028181+0000","last_peered":"2024-09-21T23:18:51.028181+0000","last_clean":"2024-09-21T23:18:51.028181+0000","last_became_active":"2024-09-21T23:18:46.786927+0000","last_became_peered":"2024-09-21T23:18:46.786927+0000","last_unstale":"2024-09-21T23:18:51.028181+0000","last_undegraded":"2024-09-21T23:18:51.028181+0000","last_fullsized":"2024-09-21T23:18:51.028181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:24:25.703682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003514428,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"173'472","reported_seq":4619,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788417+0000","last_change":"2024-09-21T23:18:29.962695+0000","last_active":"2024-09-21T23:18:51.788417+0000","last_peered":"2024-09-21T23:18:51.788417+0000","last_clean":"2024-09-21T23:18:51.788417+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:51.788417+0000","last_undegraded":"2024-09-21T23:18:51.788417+0000","last_fullsized":"2024-09-21T23:18:51.788417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:37.106862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073184640000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'479","reported_seq":6832,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788736+0000","last_change":"2024-09-21T23:18:36.787585+0000","last_active":"2024-09-21T23:18:51.788736+0000","last_peered":"2024-09-21T23:18:51.788736+0000","last_clean":"2024-09-21T23:18:51.788736+0000","last_became_active":"2024-09-21T23:18:36.784584+0000","last_became_peered":"2024-09-21T23:18:36.784584+0000","last_unstale":"2024-09-21T23:18:51.788736+0000","last_undegraded":"2024-09-21T23:18:51.788736+0000","last_fullsized":"2024-09-21T23:18:51.788736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:10:34.977998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080862480000000007,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"178'776","reported_seq":6199,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788078+0000","last_change":"2024-09-21T23:18:29.956286+0000","last_active":"2024-09-21T23:18:51.788078+0000","last_peered":"2024-09-21T23:18:51.788078+0000","last_clean":"2024-09-21T23:18:51.788078+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:51.788078+0000","last_undegraded":"2024-09-21T23:18:51.788078+0000","last_fullsized":"2024-09-21T23:18:51.788078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:23:05.964943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028297349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"178'539","reported_seq":5572,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027938+0000","last_change":"2024-09-21T23:18:40.806974+0000","last_active":"2024-09-21T23:18:51.027938+0000","last_peered":"2024-09-21T23:18:51.027938+0000","last_clean":"2024-09-21T23:18:51.027938+0000","last_became_active":"2024-09-21T23:18:40.802654+0000","last_became_peered":"2024-09-21T23:18:40.802654+0000","last_unstale":"2024-09-21T23:18:51.027938+0000","last_undegraded":"2024-09-21T23:18:51.027938+0000","last_fullsized":"2024-09-21T23:18:51.027938+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:51.646949+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029604689999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"177'807","reported_seq":22299,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790855+0000","last_change":"2024-09-21T23:18:40.819974+0000","last_active":"2024-09-21T23:18:51.790855+0000","last_peered":"2024-09-21T23:18:51.790855+0000","last_clean":"2024-09-21T23:18:51.790855+0000","last_became_active":"2024-09-21T23:18:40.816843+0000","last_became_peered":"2024-09-21T23:18:40.816843+0000","last_unstale":"2024-09-21T23:18:51.790855+0000","last_undegraded":"2024-09-21T23:18:51.790855+0000","last_fullsized":"2024-09-21T23:18:51.790855+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:12.452126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"179'484","reported_seq":9830,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028099+0000","last_change":"2024-09-21T23:18:38.798246+0000","last_active":"2024-09-21T23:18:51.028099+0000","last_peered":"2024-09-21T23:18:51.028099+0000","last_clean":"2024-09-21T23:18:51.028099+0000","last_became_active":"2024-09-21T23:18:38.791632+0000","last_became_peered":"2024-09-21T23:18:38.791632+0000","last_unstale":"2024-09-21T23:18:51.028099+0000","last_undegraded":"2024-09-21T23:18:51.028099+0000","last_fullsized":"2024-09-21T23:18:51.028099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:30.701459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025997469999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":527,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":330,"num_read":313855,"num_read_kb":1079349,"num_write":21231,"num_write_kb":208209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4344,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20259,"ondisk_log_size":20259,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607606,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51580,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320260,"statfs":{"total":96636764160,"available":96583946240,"internally_reserved":0,"allocated":2396160,"data_stored":1003182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55736,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316104,"statfs":{"total":96636764160,"available":96579690496,"internally_reserved":0,"allocated":2396160,"data_stored":1003182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607606,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42436,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329404,"statfs":{"total":96636764160,"available":96593309696,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607606,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2392064,"data_stored":1003163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77968,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293872,"statfs":{"total":96636764160,"available":96556924928,"internally_reserved":0,"allocated":2412544,"data_stored":1008200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005092,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":2916,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":2985984,"data_stored":1593531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:54.692 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T23:18:54.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T23:18:55.001 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T23:18:55.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T23:18:55.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:55.310 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:55.311 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:55.326 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-21T23:18:54.910579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":338,"num_object_clones":0,"num_object_copies":676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":330,"num_read":313971,"num_read_kb":1079449,"num_write":21392,"num_write_kb":210627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4354,"num_bytes_recovered":24942653,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20379,"ondisk_log_size":20379,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340952,"kb_used_data":15240,"kb_used_omap":13,"kb_used_meta":325554,"kb_avail":565890088,"statfs":{"total":579820584960,"available":579471450112,"internally_reserved":0,"allocated":15605760,"data_stored":7219786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13824,"internal_metadata":333367808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-550,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.127578"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13668,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790048+0000","last_change":"2024-09-21T23:18:40.802504+0000","last_active":"2024-09-21T23:18:51.790048+0000","last_peered":"2024-09-21T23:18:51.790048+0000","last_clean":"2024-09-21T23:18:51.790048+0000","last_became_active":"2024-09-21T23:18:40.801017+0000","last_became_peered":"2024-09-21T23:18:40.801017+0000","last_unstale":"2024-09-21T23:18:51.790048+0000","last_undegraded":"2024-09-21T23:18:51.790048+0000","last_fullsized":"2024-09-21T23:18:51.790048+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:02.839217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'484","reported_seq":5924,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790724+0000","last_change":"2024-09-21T23:18:46.792370+0000","last_active":"2024-09-21T23:18:51.790724+0000","last_peered":"2024-09-21T23:18:51.790724+0000","last_clean":"2024-09-21T23:18:51.790724+0000","last_became_active":"2024-09-21T23:18:46.789102+0000","last_became_peered":"2024-09-21T23:18:46.789102+0000","last_unstale":"2024-09-21T23:18:51.790724+0000","last_undegraded":"2024-09-21T23:18:51.790724+0000","last_fullsized":"2024-09-21T23:18:51.790724+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:39.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020600919999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"177'597","reported_seq":8089,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788065+0000","last_change":"2024-09-21T23:18:29.959111+0000","last_active":"2024-09-21T23:18:51.788065+0000","last_peered":"2024-09-21T23:18:51.788065+0000","last_clean":"2024-09-21T23:18:51.788065+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:51.788065+0000","last_undegraded":"2024-09-21T23:18:51.788065+0000","last_fullsized":"2024-09-21T23:18:51.788065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:19.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053022649999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"179'1015","reported_seq":13065,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787713+0000","last_change":"2024-09-21T23:18:29.959245+0000","last_active":"2024-09-21T23:18:51.787713+0000","last_peered":"2024-09-21T23:18:51.787713+0000","last_clean":"2024-09-21T23:18:51.787713+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:51.787713+0000","last_undegraded":"2024-09-21T23:18:51.787713+0000","last_fullsized":"2024-09-21T23:18:51.787713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:45.242522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048112889999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17738,"num_read_kb":71148,"num_write":1049,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'490","reported_seq":5328,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791127+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:51.791127+0000","last_peered":"2024-09-21T23:18:51.791127+0000","last_clean":"2024-09-21T23:18:51.791127+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:51.791127+0000","last_undegraded":"2024-09-21T23:18:51.791127+0000","last_fullsized":"2024-09-21T23:18:51.791127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"178'696","reported_seq":10529,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787725+0000","last_change":"2024-09-21T23:18:29.954505+0000","last_active":"2024-09-21T23:18:51.787725+0000","last_peered":"2024-09-21T23:18:51.787725+0000","last_clean":"2024-09-21T23:18:51.787725+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:51.787725+0000","last_undegraded":"2024-09-21T23:18:51.787725+0000","last_fullsized":"2024-09-21T23:18:51.787725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:47.554016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059155759999999996,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12157,"num_read_kb":26818,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'411","reported_seq":5443,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029539+0000","last_change":"2024-09-21T23:18:30.016900+0000","last_active":"2024-09-21T23:18:51.029539+0000","last_peered":"2024-09-21T23:18:51.029539+0000","last_clean":"2024-09-21T23:18:51.029539+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:51.029539+0000","last_undegraded":"2024-09-21T23:18:51.029539+0000","last_fullsized":"2024-09-21T23:18:51.029539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:28.556199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016355880000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'672","reported_seq":18660,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790494+0000","last_change":"2024-09-21T23:18:47.800021+0000","last_active":"2024-09-21T23:18:51.790494+0000","last_peered":"2024-09-21T23:18:51.790494+0000","last_clean":"2024-09-21T23:18:51.790494+0000","last_became_active":"2024-09-21T23:18:47.797088+0000","last_became_peered":"2024-09-21T23:18:47.797088+0000","last_unstale":"2024-09-21T23:18:51.790494+0000","last_undegraded":"2024-09-21T23:18:51.790494+0000","last_fullsized":"2024-09-21T23:18:51.790494+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:31.792480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"178'416","reported_seq":13674,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790017+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:51.790017+0000","last_peered":"2024-09-21T23:18:51.790017+0000","last_clean":"2024-09-21T23:18:51.790017+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:51.790017+0000","last_undegraded":"2024-09-21T23:18:51.790017+0000","last_fullsized":"2024-09-21T23:18:51.790017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"178'581","reported_seq":6245,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788309+0000","last_change":"2024-09-21T23:18:39.802796+0000","last_active":"2024-09-21T23:18:51.788309+0000","last_peered":"2024-09-21T23:18:51.788309+0000","last_clean":"2024-09-21T23:18:51.788309+0000","last_became_active":"2024-09-21T23:18:39.798603+0000","last_became_peered":"2024-09-21T23:18:39.798603+0000","last_unstale":"2024-09-21T23:18:51.788309+0000","last_undegraded":"2024-09-21T23:18:51.788309+0000","last_fullsized":"2024-09-21T23:18:51.788309+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:00:20.739193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061281979999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"177'349","reported_seq":4291,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028231+0000","last_change":"2024-09-21T23:18:38.796650+0000","last_active":"2024-09-21T23:18:51.028231+0000","last_peered":"2024-09-21T23:18:51.028231+0000","last_clean":"2024-09-21T23:18:51.028231+0000","last_became_active":"2024-09-21T23:18:38.791983+0000","last_became_peered":"2024-09-21T23:18:38.791983+0000","last_unstale":"2024-09-21T23:18:51.028231+0000","last_undegraded":"2024-09-21T23:18:51.028231+0000","last_fullsized":"2024-09-21T23:18:51.028231+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:55.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"178'938","reported_seq":8298,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028293+0000","last_change":"2024-09-21T23:18:39.806128+0000","last_active":"2024-09-21T23:18:51.028293+0000","last_peered":"2024-09-21T23:18:51.028293+0000","last_clean":"2024-09-21T23:18:51.028293+0000","last_became_active":"2024-09-21T23:18:39.801495+0000","last_became_peered":"2024-09-21T23:18:39.801495+0000","last_unstale":"2024-09-21T23:18:51.028293+0000","last_undegraded":"2024-09-21T23:18:51.028293+0000","last_fullsized":"2024-09-21T23:18:51.028293+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:08.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074401700000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028633+0000","last_change":"2024-09-21T23:18:33.987293+0000","last_active":"2024-09-21T23:18:51.028633+0000","last_peered":"2024-09-21T23:18:51.028633+0000","last_clean":"2024-09-21T23:18:51.028633+0000","last_became_active":"2024-09-21T23:18:33.986262+0000","last_became_peered":"2024-09-21T23:18:33.986262+0000","last_unstale":"2024-09-21T23:18:51.028633+0000","last_undegraded":"2024-09-21T23:18:51.028633+0000","last_fullsized":"2024-09-21T23:18:51.028633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:53.298589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20288,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790013+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:51.790013+0000","last_peered":"2024-09-21T23:18:51.790013+0000","last_clean":"2024-09-21T23:18:51.790013+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:51.790013+0000","last_undegraded":"2024-09-21T23:18:51.790013+0000","last_fullsized":"2024-09-21T23:18:51.790013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029619+0000","last_change":"2024-09-21T23:18:33.986624+0000","last_active":"2024-09-21T23:18:51.029619+0000","last_peered":"2024-09-21T23:18:51.029619+0000","last_clean":"2024-09-21T23:18:51.029619+0000","last_became_active":"2024-09-21T23:18:33.986227+0000","last_became_peered":"2024-09-21T23:18:33.986227+0000","last_unstale":"2024-09-21T23:18:51.029619+0000","last_undegraded":"2024-09-21T23:18:51.029619+0000","last_fullsized":"2024-09-21T23:18:51.029619+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:46.791389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8599,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790716+0000","last_change":"2024-09-21T23:18:48.797459+0000","last_active":"2024-09-21T23:18:51.790716+0000","last_peered":"2024-09-21T23:18:51.790716+0000","last_clean":"2024-09-21T23:18:51.790716+0000","last_became_active":"2024-09-21T23:18:48.795281+0000","last_became_peered":"2024-09-21T23:18:48.795281+0000","last_unstale":"2024-09-21T23:18:51.790716+0000","last_undegraded":"2024-09-21T23:18:51.790716+0000","last_fullsized":"2024-09-21T23:18:51.790716+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:12:45.760380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244768,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028468+0000","last_change":"2024-09-21T23:18:33.982766+0000","last_active":"2024-09-21T23:18:51.028468+0000","last_peered":"2024-09-21T23:18:51.028468+0000","last_clean":"2024-09-21T23:18:51.028468+0000","last_became_active":"2024-09-21T23:18:33.981926+0000","last_became_peered":"2024-09-21T23:18:33.981926+0000","last_unstale":"2024-09-21T23:18:51.028468+0000","last_undegraded":"2024-09-21T23:18:51.028468+0000","last_fullsized":"2024-09-21T23:18:51.028468+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:06.668878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":3308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029990+0000","last_change":"2024-09-21T23:18:47.796589+0000","last_active":"2024-09-21T23:18:51.029990+0000","last_peered":"2024-09-21T23:18:51.029990+0000","last_clean":"2024-09-21T23:18:51.029990+0000","last_became_active":"2024-09-21T23:18:47.794405+0000","last_became_peered":"2024-09-21T23:18:47.794405+0000","last_unstale":"2024-09-21T23:18:51.029990+0000","last_undegraded":"2024-09-21T23:18:51.029990+0000","last_fullsized":"2024-09-21T23:18:51.029990+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:07:32.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073231999999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":269,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029959+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:51.029959+0000","last_peered":"2024-09-21T23:18:51.029959+0000","last_clean":"2024-09-21T23:18:51.029959+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:51.029959+0000","last_undegraded":"2024-09-21T23:18:51.029959+0000","last_fullsized":"2024-09-21T23:18:51.029959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":11175,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788244+0000","last_change":"2024-09-21T23:18:42.790134+0000","last_active":"2024-09-21T23:18:51.788244+0000","last_peered":"2024-09-21T23:18:51.788244+0000","last_clean":"2024-09-21T23:18:51.788244+0000","last_became_active":"2024-09-21T23:18:42.787666+0000","last_became_peered":"2024-09-21T23:18:42.787666+0000","last_unstale":"2024-09-21T23:18:51.788244+0000","last_undegraded":"2024-09-21T23:18:51.788244+0000","last_fullsized":"2024-09-21T23:18:51.788244+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:20.552688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069917950000000003,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'792","reported_seq":7120,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787876+0000","last_change":"2024-09-21T23:18:29.953189+0000","last_active":"2024-09-21T23:18:51.787876+0000","last_peered":"2024-09-21T23:18:51.787876+0000","last_clean":"2024-09-21T23:18:51.787876+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:51.787876+0000","last_undegraded":"2024-09-21T23:18:51.787876+0000","last_fullsized":"2024-09-21T23:18:51.787876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:20.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046990039999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8412,"num_read_kb":29133,"num_write":771,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":404,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787751+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:51.787751+0000","last_peered":"2024-09-21T23:18:51.787751+0000","last_clean":"2024-09-21T23:18:51.787751+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:51.787751+0000","last_undegraded":"2024-09-21T23:18:51.787751+0000","last_fullsized":"2024-09-21T23:18:51.787751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6682,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029845+0000","last_change":"2024-09-21T23:18:47.800836+0000","last_active":"2024-09-21T23:18:51.029845+0000","last_peered":"2024-09-21T23:18:51.029845+0000","last_clean":"2024-09-21T23:18:51.029845+0000","last_became_active":"2024-09-21T23:18:47.798894+0000","last_became_peered":"2024-09-21T23:18:47.798894+0000","last_unstale":"2024-09-21T23:18:51.029845+0000","last_undegraded":"2024-09-21T23:18:51.029845+0000","last_fullsized":"2024-09-21T23:18:51.029845+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:19.199753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015079399999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790577+0000","last_change":"2024-09-21T23:18:42.788361+0000","last_active":"2024-09-21T23:18:51.790577+0000","last_peered":"2024-09-21T23:18:51.790577+0000","last_clean":"2024-09-21T23:18:51.790577+0000","last_became_active":"2024-09-21T23:18:42.788015+0000","last_became_peered":"2024-09-21T23:18:42.788015+0000","last_unstale":"2024-09-21T23:18:51.790577+0000","last_undegraded":"2024-09-21T23:18:51.790577+0000","last_fullsized":"2024-09-21T23:18:51.790577+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:44.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4792,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790731+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:51.790731+0000","last_peered":"2024-09-21T23:18:51.790731+0000","last_clean":"2024-09-21T23:18:51.790731+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:51.790731+0000","last_undegraded":"2024-09-21T23:18:51.790731+0000","last_fullsized":"2024-09-21T23:18:51.790731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027766+0000","last_change":"2024-09-21T23:18:33.978285+0000","last_active":"2024-09-21T23:18:51.027766+0000","last_peered":"2024-09-21T23:18:51.027766+0000","last_clean":"2024-09-21T23:18:51.027766+0000","last_became_active":"2024-09-21T23:18:33.977449+0000","last_became_peered":"2024-09-21T23:18:33.977449+0000","last_unstale":"2024-09-21T23:18:51.027766+0000","last_undegraded":"2024-09-21T23:18:51.027766+0000","last_fullsized":"2024-09-21T23:18:51.027766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:03.716381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":17031,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029958+0000","last_change":"2024-09-21T23:18:30.017566+0000","last_active":"2024-09-21T23:18:51.029958+0000","last_peered":"2024-09-21T23:18:51.029958+0000","last_clean":"2024-09-21T23:18:51.029958+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:51.029958+0000","last_undegraded":"2024-09-21T23:18:51.029958+0000","last_fullsized":"2024-09-21T23:18:51.029958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:56.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020263540000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788003+0000","last_change":"2024-09-21T23:18:42.793227+0000","last_active":"2024-09-21T23:18:51.788003+0000","last_peered":"2024-09-21T23:18:51.788003+0000","last_clean":"2024-09-21T23:18:51.788003+0000","last_became_active":"2024-09-21T23:18:42.788526+0000","last_became_peered":"2024-09-21T23:18:42.788526+0000","last_unstale":"2024-09-21T23:18:51.788003+0000","last_undegraded":"2024-09-21T23:18:51.788003+0000","last_fullsized":"2024-09-21T23:18:51.788003+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:53.455486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":483,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027862+0000","last_change":"2024-09-21T23:18:36.778564+0000","last_active":"2024-09-21T23:18:51.027862+0000","last_peered":"2024-09-21T23:18:51.027862+0000","last_clean":"2024-09-21T23:18:51.027862+0000","last_became_active":"2024-09-21T23:18:36.777596+0000","last_became_peered":"2024-09-21T23:18:36.777596+0000","last_unstale":"2024-09-21T23:18:51.027862+0000","last_undegraded":"2024-09-21T23:18:51.027862+0000","last_fullsized":"2024-09-21T23:18:51.027862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:32.921245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12928,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790372+0000","last_change":"2024-09-21T23:18:47.797129+0000","last_active":"2024-09-21T23:18:51.790372+0000","last_peered":"2024-09-21T23:18:51.790372+0000","last_clean":"2024-09-21T23:18:51.790372+0000","last_became_active":"2024-09-21T23:18:47.793978+0000","last_became_peered":"2024-09-21T23:18:47.793978+0000","last_unstale":"2024-09-21T23:18:51.790372+0000","last_undegraded":"2024-09-21T23:18:51.790372+0000","last_fullsized":"2024-09-21T23:18:51.790372+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:42.993883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011535569999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"178'592","reported_seq":5301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787636+0000","last_change":"2024-09-21T23:18:46.790171+0000","last_active":"2024-09-21T23:18:51.787636+0000","last_peered":"2024-09-21T23:18:51.787636+0000","last_clean":"2024-09-21T23:18:51.787636+0000","last_became_active":"2024-09-21T23:18:46.787797+0000","last_became_peered":"2024-09-21T23:18:46.787797+0000","last_unstale":"2024-09-21T23:18:51.787636+0000","last_undegraded":"2024-09-21T23:18:51.787636+0000","last_fullsized":"2024-09-21T23:18:51.787636+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:43:38.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085867570000000004,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"178'354","reported_seq":4260,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027978+0000","last_change":"2024-09-21T23:18:38.801696+0000","last_active":"2024-09-21T23:18:51.027978+0000","last_peered":"2024-09-21T23:18:51.027978+0000","last_clean":"2024-09-21T23:18:51.027978+0000","last_became_active":"2024-09-21T23:18:38.795761+0000","last_became_peered":"2024-09-21T23:18:38.795761+0000","last_unstale":"2024-09-21T23:18:51.027978+0000","last_undegraded":"2024-09-21T23:18:51.027978+0000","last_fullsized":"2024-09-21T23:18:51.027978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:33.754942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"177'791","reported_seq":8669,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791035+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:51.791035+0000","last_peered":"2024-09-21T23:18:51.791035+0000","last_clean":"2024-09-21T23:18:51.791035+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:51.791035+0000","last_undegraded":"2024-09-21T23:18:51.791035+0000","last_fullsized":"2024-09-21T23:18:51.791035+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"179'686","reported_seq":6156,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029573+0000","last_change":"2024-09-21T23:18:42.790726+0000","last_active":"2024-09-21T23:18:51.029573+0000","last_peered":"2024-09-21T23:18:51.029573+0000","last_clean":"2024-09-21T23:18:51.029573+0000","last_became_active":"2024-09-21T23:18:42.788002+0000","last_became_peered":"2024-09-21T23:18:42.788002+0000","last_unstale":"2024-09-21T23:18:51.029573+0000","last_undegraded":"2024-09-21T23:18:51.029573+0000","last_fullsized":"2024-09-21T23:18:51.029573+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:04:27.488128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062598719999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7976,"num_read_kb":18238,"num_write":516,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"177'475","reported_seq":5551,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028181+0000","last_change":"2024-09-21T23:18:46.790120+0000","last_active":"2024-09-21T23:18:51.028181+0000","last_peered":"2024-09-21T23:18:51.028181+0000","last_clean":"2024-09-21T23:18:51.028181+0000","last_became_active":"2024-09-21T23:18:46.786927+0000","last_became_peered":"2024-09-21T23:18:46.786927+0000","last_unstale":"2024-09-21T23:18:51.028181+0000","last_undegraded":"2024-09-21T23:18:51.028181+0000","last_fullsized":"2024-09-21T23:18:51.028181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:24:25.703682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003514428,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"173'472","reported_seq":4619,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788417+0000","last_change":"2024-09-21T23:18:29.962695+0000","last_active":"2024-09-21T23:18:51.788417+0000","last_peered":"2024-09-21T23:18:51.788417+0000","last_clean":"2024-09-21T23:18:51.788417+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:51.788417+0000","last_undegraded":"2024-09-21T23:18:51.788417+0000","last_fullsized":"2024-09-21T23:18:51.788417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:37.106862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073184640000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'479","reported_seq":6832,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788736+0000","last_change":"2024-09-21T23:18:36.787585+0000","last_active":"2024-09-21T23:18:51.788736+0000","last_peered":"2024-09-21T23:18:51.788736+0000","last_clean":"2024-09-21T23:18:51.788736+0000","last_became_active":"2024-09-21T23:18:36.784584+0000","last_became_peered":"2024-09-21T23:18:36.784584+0000","last_unstale":"2024-09-21T23:18:51.788736+0000","last_undegraded":"2024-09-21T23:18:51.788736+0000","last_fullsized":"2024-09-21T23:18:51.788736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:10:34.977998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080862480000000007,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"178'776","reported_seq":6199,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788078+0000","last_change":"2024-09-21T23:18:29.956286+0000","last_active":"2024-09-21T23:18:51.788078+0000","last_peered":"2024-09-21T23:18:51.788078+0000","last_clean":"2024-09-21T23:18:51.788078+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:51.788078+0000","last_undegraded":"2024-09-21T23:18:51.788078+0000","last_fullsized":"2024-09-21T23:18:51.788078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:23:05.964943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028297349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"178'539","reported_seq":5572,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027938+0000","last_change":"2024-09-21T23:18:40.806974+0000","last_active":"2024-09-21T23:18:51.027938+0000","last_peered":"2024-09-21T23:18:51.027938+0000","last_clean":"2024-09-21T23:18:51.027938+0000","last_became_active":"2024-09-21T23:18:40.802654+0000","last_became_peered":"2024-09-21T23:18:40.802654+0000","last_unstale":"2024-09-21T23:18:51.027938+0000","last_undegraded":"2024-09-21T23:18:51.027938+0000","last_fullsized":"2024-09-21T23:18:51.027938+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:51.646949+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029604689999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"177'807","reported_seq":22299,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790855+0000","last_change":"2024-09-21T23:18:40.819974+0000","last_active":"2024-09-21T23:18:51.790855+0000","last_peered":"2024-09-21T23:18:51.790855+0000","last_clean":"2024-09-21T23:18:51.790855+0000","last_became_active":"2024-09-21T23:18:40.816843+0000","last_became_peered":"2024-09-21T23:18:40.816843+0000","last_unstale":"2024-09-21T23:18:51.790855+0000","last_undegraded":"2024-09-21T23:18:51.790855+0000","last_fullsized":"2024-09-21T23:18:51.790855+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:12.452126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"179'484","reported_seq":9830,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028099+0000","last_change":"2024-09-21T23:18:38.798246+0000","last_active":"2024-09-21T23:18:51.028099+0000","last_peered":"2024-09-21T23:18:51.028099+0000","last_clean":"2024-09-21T23:18:51.028099+0000","last_became_active":"2024-09-21T23:18:38.791632+0000","last_became_peered":"2024-09-21T23:18:38.791632+0000","last_unstale":"2024-09-21T23:18:51.028099+0000","last_undegraded":"2024-09-21T23:18:51.028099+0000","last_fullsized":"2024-09-21T23:18:51.028099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:30.701459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025997469999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":527,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":330,"num_read":313855,"num_read_kb":1079349,"num_write":21231,"num_write_kb":208209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4344,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20259,"ondisk_log_size":20259,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51596,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320244,"statfs":{"total":96636764160,"available":96583929856,"internally_reserved":0,"allocated":2408448,"data_stored":1008181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55736,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316104,"statfs":{"total":96636764160,"available":96579690496,"internally_reserved":0,"allocated":2396160,"data_stored":1003182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607607,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42436,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329404,"statfs":{"total":96636764160,"available":96593309696,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51588,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320252,"statfs":{"total":96636764160,"available":96583938048,"internally_reserved":0,"allocated":2404352,"data_stored":1008162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607608,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77968,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293872,"statfs":{"total":96636764160,"available":96556924928,"internally_reserved":0,"allocated":2412544,"data_stored":1008200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005092,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":2916,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":2985984,"data_stored":1593531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:55.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:55.627 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:55.643 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-21T23:18:54.910579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":338,"num_object_clones":0,"num_object_copies":676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":330,"num_read":313971,"num_read_kb":1079449,"num_write":21392,"num_write_kb":210627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4354,"num_bytes_recovered":24942653,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20379,"ondisk_log_size":20379,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340952,"kb_used_data":15240,"kb_used_omap":13,"kb_used_meta":325554,"kb_avail":565890088,"statfs":{"total":579820584960,"available":579471450112,"internally_reserved":0,"allocated":15605760,"data_stored":7219786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13824,"internal_metadata":333367808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-550,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.127578"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13668,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790048+0000","last_change":"2024-09-21T23:18:40.802504+0000","last_active":"2024-09-21T23:18:51.790048+0000","last_peered":"2024-09-21T23:18:51.790048+0000","last_clean":"2024-09-21T23:18:51.790048+0000","last_became_active":"2024-09-21T23:18:40.801017+0000","last_became_peered":"2024-09-21T23:18:40.801017+0000","last_unstale":"2024-09-21T23:18:51.790048+0000","last_undegraded":"2024-09-21T23:18:51.790048+0000","last_fullsized":"2024-09-21T23:18:51.790048+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:02.839217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'484","reported_seq":5924,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790724+0000","last_change":"2024-09-21T23:18:46.792370+0000","last_active":"2024-09-21T23:18:51.790724+0000","last_peered":"2024-09-21T23:18:51.790724+0000","last_clean":"2024-09-21T23:18:51.790724+0000","last_became_active":"2024-09-21T23:18:46.789102+0000","last_became_peered":"2024-09-21T23:18:46.789102+0000","last_unstale":"2024-09-21T23:18:51.790724+0000","last_undegraded":"2024-09-21T23:18:51.790724+0000","last_fullsized":"2024-09-21T23:18:51.790724+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:39.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020600919999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"177'597","reported_seq":8089,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788065+0000","last_change":"2024-09-21T23:18:29.959111+0000","last_active":"2024-09-21T23:18:51.788065+0000","last_peered":"2024-09-21T23:18:51.788065+0000","last_clean":"2024-09-21T23:18:51.788065+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:51.788065+0000","last_undegraded":"2024-09-21T23:18:51.788065+0000","last_fullsized":"2024-09-21T23:18:51.788065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:19.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053022649999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"179'1015","reported_seq":13065,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787713+0000","last_change":"2024-09-21T23:18:29.959245+0000","last_active":"2024-09-21T23:18:51.787713+0000","last_peered":"2024-09-21T23:18:51.787713+0000","last_clean":"2024-09-21T23:18:51.787713+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:51.787713+0000","last_undegraded":"2024-09-21T23:18:51.787713+0000","last_fullsized":"2024-09-21T23:18:51.787713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:45.242522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048112889999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17738,"num_read_kb":71148,"num_write":1049,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'490","reported_seq":5328,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791127+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:51.791127+0000","last_peered":"2024-09-21T23:18:51.791127+0000","last_clean":"2024-09-21T23:18:51.791127+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:51.791127+0000","last_undegraded":"2024-09-21T23:18:51.791127+0000","last_fullsized":"2024-09-21T23:18:51.791127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"178'696","reported_seq":10529,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787725+0000","last_change":"2024-09-21T23:18:29.954505+0000","last_active":"2024-09-21T23:18:51.787725+0000","last_peered":"2024-09-21T23:18:51.787725+0000","last_clean":"2024-09-21T23:18:51.787725+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:51.787725+0000","last_undegraded":"2024-09-21T23:18:51.787725+0000","last_fullsized":"2024-09-21T23:18:51.787725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:47.554016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059155759999999996,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12157,"num_read_kb":26818,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'411","reported_seq":5443,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029539+0000","last_change":"2024-09-21T23:18:30.016900+0000","last_active":"2024-09-21T23:18:51.029539+0000","last_peered":"2024-09-21T23:18:51.029539+0000","last_clean":"2024-09-21T23:18:51.029539+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:51.029539+0000","last_undegraded":"2024-09-21T23:18:51.029539+0000","last_fullsized":"2024-09-21T23:18:51.029539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:28.556199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016355880000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'672","reported_seq":18660,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790494+0000","last_change":"2024-09-21T23:18:47.800021+0000","last_active":"2024-09-21T23:18:51.790494+0000","last_peered":"2024-09-21T23:18:51.790494+0000","last_clean":"2024-09-21T23:18:51.790494+0000","last_became_active":"2024-09-21T23:18:47.797088+0000","last_became_peered":"2024-09-21T23:18:47.797088+0000","last_unstale":"2024-09-21T23:18:51.790494+0000","last_undegraded":"2024-09-21T23:18:51.790494+0000","last_fullsized":"2024-09-21T23:18:51.790494+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:31.792480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"178'416","reported_seq":13674,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790017+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:51.790017+0000","last_peered":"2024-09-21T23:18:51.790017+0000","last_clean":"2024-09-21T23:18:51.790017+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:51.790017+0000","last_undegraded":"2024-09-21T23:18:51.790017+0000","last_fullsized":"2024-09-21T23:18:51.790017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"178'581","reported_seq":6245,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788309+0000","last_change":"2024-09-21T23:18:39.802796+0000","last_active":"2024-09-21T23:18:51.788309+0000","last_peered":"2024-09-21T23:18:51.788309+0000","last_clean":"2024-09-21T23:18:51.788309+0000","last_became_active":"2024-09-21T23:18:39.798603+0000","last_became_peered":"2024-09-21T23:18:39.798603+0000","last_unstale":"2024-09-21T23:18:51.788309+0000","last_undegraded":"2024-09-21T23:18:51.788309+0000","last_fullsized":"2024-09-21T23:18:51.788309+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:00:20.739193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061281979999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"177'349","reported_seq":4291,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028231+0000","last_change":"2024-09-21T23:18:38.796650+0000","last_active":"2024-09-21T23:18:51.028231+0000","last_peered":"2024-09-21T23:18:51.028231+0000","last_clean":"2024-09-21T23:18:51.028231+0000","last_became_active":"2024-09-21T23:18:38.791983+0000","last_became_peered":"2024-09-21T23:18:38.791983+0000","last_unstale":"2024-09-21T23:18:51.028231+0000","last_undegraded":"2024-09-21T23:18:51.028231+0000","last_fullsized":"2024-09-21T23:18:51.028231+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:55.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"178'938","reported_seq":8298,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028293+0000","last_change":"2024-09-21T23:18:39.806128+0000","last_active":"2024-09-21T23:18:51.028293+0000","last_peered":"2024-09-21T23:18:51.028293+0000","last_clean":"2024-09-21T23:18:51.028293+0000","last_became_active":"2024-09-21T23:18:39.801495+0000","last_became_peered":"2024-09-21T23:18:39.801495+0000","last_unstale":"2024-09-21T23:18:51.028293+0000","last_undegraded":"2024-09-21T23:18:51.028293+0000","last_fullsized":"2024-09-21T23:18:51.028293+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:08.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074401700000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028633+0000","last_change":"2024-09-21T23:18:33.987293+0000","last_active":"2024-09-21T23:18:51.028633+0000","last_peered":"2024-09-21T23:18:51.028633+0000","last_clean":"2024-09-21T23:18:51.028633+0000","last_became_active":"2024-09-21T23:18:33.986262+0000","last_became_peered":"2024-09-21T23:18:33.986262+0000","last_unstale":"2024-09-21T23:18:51.028633+0000","last_undegraded":"2024-09-21T23:18:51.028633+0000","last_fullsized":"2024-09-21T23:18:51.028633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:53.298589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20288,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790013+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:51.790013+0000","last_peered":"2024-09-21T23:18:51.790013+0000","last_clean":"2024-09-21T23:18:51.790013+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:51.790013+0000","last_undegraded":"2024-09-21T23:18:51.790013+0000","last_fullsized":"2024-09-21T23:18:51.790013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029619+0000","last_change":"2024-09-21T23:18:33.986624+0000","last_active":"2024-09-21T23:18:51.029619+0000","last_peered":"2024-09-21T23:18:51.029619+0000","last_clean":"2024-09-21T23:18:51.029619+0000","last_became_active":"2024-09-21T23:18:33.986227+0000","last_became_peered":"2024-09-21T23:18:33.986227+0000","last_unstale":"2024-09-21T23:18:51.029619+0000","last_undegraded":"2024-09-21T23:18:51.029619+0000","last_fullsized":"2024-09-21T23:18:51.029619+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:46.791389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8599,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790716+0000","last_change":"2024-09-21T23:18:48.797459+0000","last_active":"2024-09-21T23:18:51.790716+0000","last_peered":"2024-09-21T23:18:51.790716+0000","last_clean":"2024-09-21T23:18:51.790716+0000","last_became_active":"2024-09-21T23:18:48.795281+0000","last_became_peered":"2024-09-21T23:18:48.795281+0000","last_unstale":"2024-09-21T23:18:51.790716+0000","last_undegraded":"2024-09-21T23:18:51.790716+0000","last_fullsized":"2024-09-21T23:18:51.790716+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:12:45.760380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244768,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028468+0000","last_change":"2024-09-21T23:18:33.982766+0000","last_active":"2024-09-21T23:18:51.028468+0000","last_peered":"2024-09-21T23:18:51.028468+0000","last_clean":"2024-09-21T23:18:51.028468+0000","last_became_active":"2024-09-21T23:18:33.981926+0000","last_became_peered":"2024-09-21T23:18:33.981926+0000","last_unstale":"2024-09-21T23:18:51.028468+0000","last_undegraded":"2024-09-21T23:18:51.028468+0000","last_fullsized":"2024-09-21T23:18:51.028468+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:06.668878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":3308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029990+0000","last_change":"2024-09-21T23:18:47.796589+0000","last_active":"2024-09-21T23:18:51.029990+0000","last_peered":"2024-09-21T23:18:51.029990+0000","last_clean":"2024-09-21T23:18:51.029990+0000","last_became_active":"2024-09-21T23:18:47.794405+0000","last_became_peered":"2024-09-21T23:18:47.794405+0000","last_unstale":"2024-09-21T23:18:51.029990+0000","last_undegraded":"2024-09-21T23:18:51.029990+0000","last_fullsized":"2024-09-21T23:18:51.029990+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:07:32.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073231999999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":269,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029959+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:51.029959+0000","last_peered":"2024-09-21T23:18:51.029959+0000","last_clean":"2024-09-21T23:18:51.029959+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:51.029959+0000","last_undegraded":"2024-09-21T23:18:51.029959+0000","last_fullsized":"2024-09-21T23:18:51.029959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":11175,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788244+0000","last_change":"2024-09-21T23:18:42.790134+0000","last_active":"2024-09-21T23:18:51.788244+0000","last_peered":"2024-09-21T23:18:51.788244+0000","last_clean":"2024-09-21T23:18:51.788244+0000","last_became_active":"2024-09-21T23:18:42.787666+0000","last_became_peered":"2024-09-21T23:18:42.787666+0000","last_unstale":"2024-09-21T23:18:51.788244+0000","last_undegraded":"2024-09-21T23:18:51.788244+0000","last_fullsized":"2024-09-21T23:18:51.788244+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:20.552688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069917950000000003,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'792","reported_seq":7120,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787876+0000","last_change":"2024-09-21T23:18:29.953189+0000","last_active":"2024-09-21T23:18:51.787876+0000","last_peered":"2024-09-21T23:18:51.787876+0000","last_clean":"2024-09-21T23:18:51.787876+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:51.787876+0000","last_undegraded":"2024-09-21T23:18:51.787876+0000","last_fullsized":"2024-09-21T23:18:51.787876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:20.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046990039999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8412,"num_read_kb":29133,"num_write":771,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":404,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787751+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:51.787751+0000","last_peered":"2024-09-21T23:18:51.787751+0000","last_clean":"2024-09-21T23:18:51.787751+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:51.787751+0000","last_undegraded":"2024-09-21T23:18:51.787751+0000","last_fullsized":"2024-09-21T23:18:51.787751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6682,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029845+0000","last_change":"2024-09-21T23:18:47.800836+0000","last_active":"2024-09-21T23:18:51.029845+0000","last_peered":"2024-09-21T23:18:51.029845+0000","last_clean":"2024-09-21T23:18:51.029845+0000","last_became_active":"2024-09-21T23:18:47.798894+0000","last_became_peered":"2024-09-21T23:18:47.798894+0000","last_unstale":"2024-09-21T23:18:51.029845+0000","last_undegraded":"2024-09-21T23:18:51.029845+0000","last_fullsized":"2024-09-21T23:18:51.029845+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:19.199753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015079399999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790577+0000","last_change":"2024-09-21T23:18:42.788361+0000","last_active":"2024-09-21T23:18:51.790577+0000","last_peered":"2024-09-21T23:18:51.790577+0000","last_clean":"2024-09-21T23:18:51.790577+0000","last_became_active":"2024-09-21T23:18:42.788015+0000","last_became_peered":"2024-09-21T23:18:42.788015+0000","last_unstale":"2024-09-21T23:18:51.790577+0000","last_undegraded":"2024-09-21T23:18:51.790577+0000","last_fullsized":"2024-09-21T23:18:51.790577+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:44.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4792,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790731+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:51.790731+0000","last_peered":"2024-09-21T23:18:51.790731+0000","last_clean":"2024-09-21T23:18:51.790731+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:51.790731+0000","last_undegraded":"2024-09-21T23:18:51.790731+0000","last_fullsized":"2024-09-21T23:18:51.790731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027766+0000","last_change":"2024-09-21T23:18:33.978285+0000","last_active":"2024-09-21T23:18:51.027766+0000","last_peered":"2024-09-21T23:18:51.027766+0000","last_clean":"2024-09-21T23:18:51.027766+0000","last_became_active":"2024-09-21T23:18:33.977449+0000","last_became_peered":"2024-09-21T23:18:33.977449+0000","last_unstale":"2024-09-21T23:18:51.027766+0000","last_undegraded":"2024-09-21T23:18:51.027766+0000","last_fullsized":"2024-09-21T23:18:51.027766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:03.716381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":17031,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029958+0000","last_change":"2024-09-21T23:18:30.017566+0000","last_active":"2024-09-21T23:18:51.029958+0000","last_peered":"2024-09-21T23:18:51.029958+0000","last_clean":"2024-09-21T23:18:51.029958+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:51.029958+0000","last_undegraded":"2024-09-21T23:18:51.029958+0000","last_fullsized":"2024-09-21T23:18:51.029958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:56.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020263540000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788003+0000","last_change":"2024-09-21T23:18:42.793227+0000","last_active":"2024-09-21T23:18:51.788003+0000","last_peered":"2024-09-21T23:18:51.788003+0000","last_clean":"2024-09-21T23:18:51.788003+0000","last_became_active":"2024-09-21T23:18:42.788526+0000","last_became_peered":"2024-09-21T23:18:42.788526+0000","last_unstale":"2024-09-21T23:18:51.788003+0000","last_undegraded":"2024-09-21T23:18:51.788003+0000","last_fullsized":"2024-09-21T23:18:51.788003+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:53.455486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":483,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027862+0000","last_change":"2024-09-21T23:18:36.778564+0000","last_active":"2024-09-21T23:18:51.027862+0000","last_peered":"2024-09-21T23:18:51.027862+0000","last_clean":"2024-09-21T23:18:51.027862+0000","last_became_active":"2024-09-21T23:18:36.777596+0000","last_became_peered":"2024-09-21T23:18:36.777596+0000","last_unstale":"2024-09-21T23:18:51.027862+0000","last_undegraded":"2024-09-21T23:18:51.027862+0000","last_fullsized":"2024-09-21T23:18:51.027862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:32.921245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12928,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790372+0000","last_change":"2024-09-21T23:18:47.797129+0000","last_active":"2024-09-21T23:18:51.790372+0000","last_peered":"2024-09-21T23:18:51.790372+0000","last_clean":"2024-09-21T23:18:51.790372+0000","last_became_active":"2024-09-21T23:18:47.793978+0000","last_became_peered":"2024-09-21T23:18:47.793978+0000","last_unstale":"2024-09-21T23:18:51.790372+0000","last_undegraded":"2024-09-21T23:18:51.790372+0000","last_fullsized":"2024-09-21T23:18:51.790372+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:42.993883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011535569999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"178'592","reported_seq":5301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787636+0000","last_change":"2024-09-21T23:18:46.790171+0000","last_active":"2024-09-21T23:18:51.787636+0000","last_peered":"2024-09-21T23:18:51.787636+0000","last_clean":"2024-09-21T23:18:51.787636+0000","last_became_active":"2024-09-21T23:18:46.787797+0000","last_became_peered":"2024-09-21T23:18:46.787797+0000","last_unstale":"2024-09-21T23:18:51.787636+0000","last_undegraded":"2024-09-21T23:18:51.787636+0000","last_fullsized":"2024-09-21T23:18:51.787636+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:43:38.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085867570000000004,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"178'354","reported_seq":4260,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027978+0000","last_change":"2024-09-21T23:18:38.801696+0000","last_active":"2024-09-21T23:18:51.027978+0000","last_peered":"2024-09-21T23:18:51.027978+0000","last_clean":"2024-09-21T23:18:51.027978+0000","last_became_active":"2024-09-21T23:18:38.795761+0000","last_became_peered":"2024-09-21T23:18:38.795761+0000","last_unstale":"2024-09-21T23:18:51.027978+0000","last_undegraded":"2024-09-21T23:18:51.027978+0000","last_fullsized":"2024-09-21T23:18:51.027978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:33.754942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"177'791","reported_seq":8669,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791035+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:51.791035+0000","last_peered":"2024-09-21T23:18:51.791035+0000","last_clean":"2024-09-21T23:18:51.791035+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:51.791035+0000","last_undegraded":"2024-09-21T23:18:51.791035+0000","last_fullsized":"2024-09-21T23:18:51.791035+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"179'686","reported_seq":6156,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029573+0000","last_change":"2024-09-21T23:18:42.790726+0000","last_active":"2024-09-21T23:18:51.029573+0000","last_peered":"2024-09-21T23:18:51.029573+0000","last_clean":"2024-09-21T23:18:51.029573+0000","last_became_active":"2024-09-21T23:18:42.788002+0000","last_became_peered":"2024-09-21T23:18:42.788002+0000","last_unstale":"2024-09-21T23:18:51.029573+0000","last_undegraded":"2024-09-21T23:18:51.029573+0000","last_fullsized":"2024-09-21T23:18:51.029573+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:04:27.488128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062598719999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7976,"num_read_kb":18238,"num_write":516,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"177'475","reported_seq":5551,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028181+0000","last_change":"2024-09-21T23:18:46.790120+0000","last_active":"2024-09-21T23:18:51.028181+0000","last_peered":"2024-09-21T23:18:51.028181+0000","last_clean":"2024-09-21T23:18:51.028181+0000","last_became_active":"2024-09-21T23:18:46.786927+0000","last_became_peered":"2024-09-21T23:18:46.786927+0000","last_unstale":"2024-09-21T23:18:51.028181+0000","last_undegraded":"2024-09-21T23:18:51.028181+0000","last_fullsized":"2024-09-21T23:18:51.028181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:24:25.703682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003514428,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"173'472","reported_seq":4619,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788417+0000","last_change":"2024-09-21T23:18:29.962695+0000","last_active":"2024-09-21T23:18:51.788417+0000","last_peered":"2024-09-21T23:18:51.788417+0000","last_clean":"2024-09-21T23:18:51.788417+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:51.788417+0000","last_undegraded":"2024-09-21T23:18:51.788417+0000","last_fullsized":"2024-09-21T23:18:51.788417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:37.106862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073184640000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'479","reported_seq":6832,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788736+0000","last_change":"2024-09-21T23:18:36.787585+0000","last_active":"2024-09-21T23:18:51.788736+0000","last_peered":"2024-09-21T23:18:51.788736+0000","last_clean":"2024-09-21T23:18:51.788736+0000","last_became_active":"2024-09-21T23:18:36.784584+0000","last_became_peered":"2024-09-21T23:18:36.784584+0000","last_unstale":"2024-09-21T23:18:51.788736+0000","last_undegraded":"2024-09-21T23:18:51.788736+0000","last_fullsized":"2024-09-21T23:18:51.788736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:10:34.977998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080862480000000007,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"178'776","reported_seq":6199,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788078+0000","last_change":"2024-09-21T23:18:29.956286+0000","last_active":"2024-09-21T23:18:51.788078+0000","last_peered":"2024-09-21T23:18:51.788078+0000","last_clean":"2024-09-21T23:18:51.788078+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:51.788078+0000","last_undegraded":"2024-09-21T23:18:51.788078+0000","last_fullsized":"2024-09-21T23:18:51.788078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:23:05.964943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028297349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"178'539","reported_seq":5572,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027938+0000","last_change":"2024-09-21T23:18:40.806974+0000","last_active":"2024-09-21T23:18:51.027938+0000","last_peered":"2024-09-21T23:18:51.027938+0000","last_clean":"2024-09-21T23:18:51.027938+0000","last_became_active":"2024-09-21T23:18:40.802654+0000","last_became_peered":"2024-09-21T23:18:40.802654+0000","last_unstale":"2024-09-21T23:18:51.027938+0000","last_undegraded":"2024-09-21T23:18:51.027938+0000","last_fullsized":"2024-09-21T23:18:51.027938+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:51.646949+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029604689999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"177'807","reported_seq":22299,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790855+0000","last_change":"2024-09-21T23:18:40.819974+0000","last_active":"2024-09-21T23:18:51.790855+0000","last_peered":"2024-09-21T23:18:51.790855+0000","last_clean":"2024-09-21T23:18:51.790855+0000","last_became_active":"2024-09-21T23:18:40.816843+0000","last_became_peered":"2024-09-21T23:18:40.816843+0000","last_unstale":"2024-09-21T23:18:51.790855+0000","last_undegraded":"2024-09-21T23:18:51.790855+0000","last_fullsized":"2024-09-21T23:18:51.790855+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:12.452126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"179'484","reported_seq":9830,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028099+0000","last_change":"2024-09-21T23:18:38.798246+0000","last_active":"2024-09-21T23:18:51.028099+0000","last_peered":"2024-09-21T23:18:51.028099+0000","last_clean":"2024-09-21T23:18:51.028099+0000","last_became_active":"2024-09-21T23:18:38.791632+0000","last_became_peered":"2024-09-21T23:18:38.791632+0000","last_unstale":"2024-09-21T23:18:51.028099+0000","last_undegraded":"2024-09-21T23:18:51.028099+0000","last_fullsized":"2024-09-21T23:18:51.028099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:30.701459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025997469999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":527,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":330,"num_read":313855,"num_read_kb":1079349,"num_write":21231,"num_write_kb":208209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4344,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20259,"ondisk_log_size":20259,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51596,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320244,"statfs":{"total":96636764160,"available":96583929856,"internally_reserved":0,"allocated":2408448,"data_stored":1008181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55736,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316104,"statfs":{"total":96636764160,"available":96579690496,"internally_reserved":0,"allocated":2396160,"data_stored":1003182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607607,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42436,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329404,"statfs":{"total":96636764160,"available":96593309696,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51588,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320252,"statfs":{"total":96636764160,"available":96583938048,"internally_reserved":0,"allocated":2404352,"data_stored":1008162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607608,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77968,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293872,"statfs":{"total":96636764160,"available":96556924928,"internally_reserved":0,"allocated":2412544,"data_stored":1008200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005092,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":2916,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":2985984,"data_stored":1593531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:55.644 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T23:18:55.645 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:18:55.950 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:55.951 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:18:55.969 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-21T23:18:54.910579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":338,"num_object_clones":0,"num_object_copies":676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":330,"num_read":313971,"num_read_kb":1079449,"num_write":21392,"num_write_kb":210627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4354,"num_bytes_recovered":24942653,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20379,"ondisk_log_size":20379,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340952,"kb_used_data":15240,"kb_used_omap":13,"kb_used_meta":325554,"kb_avail":565890088,"statfs":{"total":579820584960,"available":579471450112,"internally_reserved":0,"allocated":15605760,"data_stored":7219786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13824,"internal_metadata":333367808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-550,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.127578"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13668,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790048+0000","last_change":"2024-09-21T23:18:40.802504+0000","last_active":"2024-09-21T23:18:51.790048+0000","last_peered":"2024-09-21T23:18:51.790048+0000","last_clean":"2024-09-21T23:18:51.790048+0000","last_became_active":"2024-09-21T23:18:40.801017+0000","last_became_peered":"2024-09-21T23:18:40.801017+0000","last_unstale":"2024-09-21T23:18:51.790048+0000","last_undegraded":"2024-09-21T23:18:51.790048+0000","last_fullsized":"2024-09-21T23:18:51.790048+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:02.839217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'484","reported_seq":5924,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790724+0000","last_change":"2024-09-21T23:18:46.792370+0000","last_active":"2024-09-21T23:18:51.790724+0000","last_peered":"2024-09-21T23:18:51.790724+0000","last_clean":"2024-09-21T23:18:51.790724+0000","last_became_active":"2024-09-21T23:18:46.789102+0000","last_became_peered":"2024-09-21T23:18:46.789102+0000","last_unstale":"2024-09-21T23:18:51.790724+0000","last_undegraded":"2024-09-21T23:18:51.790724+0000","last_fullsized":"2024-09-21T23:18:51.790724+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:39.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020600919999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"177'597","reported_seq":8089,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788065+0000","last_change":"2024-09-21T23:18:29.959111+0000","last_active":"2024-09-21T23:18:51.788065+0000","last_peered":"2024-09-21T23:18:51.788065+0000","last_clean":"2024-09-21T23:18:51.788065+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:51.788065+0000","last_undegraded":"2024-09-21T23:18:51.788065+0000","last_fullsized":"2024-09-21T23:18:51.788065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:19.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053022649999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"179'1015","reported_seq":13065,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787713+0000","last_change":"2024-09-21T23:18:29.959245+0000","last_active":"2024-09-21T23:18:51.787713+0000","last_peered":"2024-09-21T23:18:51.787713+0000","last_clean":"2024-09-21T23:18:51.787713+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:51.787713+0000","last_undegraded":"2024-09-21T23:18:51.787713+0000","last_fullsized":"2024-09-21T23:18:51.787713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:45.242522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048112889999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17738,"num_read_kb":71148,"num_write":1049,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'490","reported_seq":5328,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791127+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:51.791127+0000","last_peered":"2024-09-21T23:18:51.791127+0000","last_clean":"2024-09-21T23:18:51.791127+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:51.791127+0000","last_undegraded":"2024-09-21T23:18:51.791127+0000","last_fullsized":"2024-09-21T23:18:51.791127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"178'696","reported_seq":10529,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787725+0000","last_change":"2024-09-21T23:18:29.954505+0000","last_active":"2024-09-21T23:18:51.787725+0000","last_peered":"2024-09-21T23:18:51.787725+0000","last_clean":"2024-09-21T23:18:51.787725+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:51.787725+0000","last_undegraded":"2024-09-21T23:18:51.787725+0000","last_fullsized":"2024-09-21T23:18:51.787725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:47.554016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059155759999999996,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12157,"num_read_kb":26818,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'411","reported_seq":5443,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029539+0000","last_change":"2024-09-21T23:18:30.016900+0000","last_active":"2024-09-21T23:18:51.029539+0000","last_peered":"2024-09-21T23:18:51.029539+0000","last_clean":"2024-09-21T23:18:51.029539+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:51.029539+0000","last_undegraded":"2024-09-21T23:18:51.029539+0000","last_fullsized":"2024-09-21T23:18:51.029539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:28.556199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016355880000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'672","reported_seq":18660,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790494+0000","last_change":"2024-09-21T23:18:47.800021+0000","last_active":"2024-09-21T23:18:51.790494+0000","last_peered":"2024-09-21T23:18:51.790494+0000","last_clean":"2024-09-21T23:18:51.790494+0000","last_became_active":"2024-09-21T23:18:47.797088+0000","last_became_peered":"2024-09-21T23:18:47.797088+0000","last_unstale":"2024-09-21T23:18:51.790494+0000","last_undegraded":"2024-09-21T23:18:51.790494+0000","last_fullsized":"2024-09-21T23:18:51.790494+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:31.792480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"178'416","reported_seq":13674,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790017+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:51.790017+0000","last_peered":"2024-09-21T23:18:51.790017+0000","last_clean":"2024-09-21T23:18:51.790017+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:51.790017+0000","last_undegraded":"2024-09-21T23:18:51.790017+0000","last_fullsized":"2024-09-21T23:18:51.790017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"178'581","reported_seq":6245,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788309+0000","last_change":"2024-09-21T23:18:39.802796+0000","last_active":"2024-09-21T23:18:51.788309+0000","last_peered":"2024-09-21T23:18:51.788309+0000","last_clean":"2024-09-21T23:18:51.788309+0000","last_became_active":"2024-09-21T23:18:39.798603+0000","last_became_peered":"2024-09-21T23:18:39.798603+0000","last_unstale":"2024-09-21T23:18:51.788309+0000","last_undegraded":"2024-09-21T23:18:51.788309+0000","last_fullsized":"2024-09-21T23:18:51.788309+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:00:20.739193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061281979999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"177'349","reported_seq":4291,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028231+0000","last_change":"2024-09-21T23:18:38.796650+0000","last_active":"2024-09-21T23:18:51.028231+0000","last_peered":"2024-09-21T23:18:51.028231+0000","last_clean":"2024-09-21T23:18:51.028231+0000","last_became_active":"2024-09-21T23:18:38.791983+0000","last_became_peered":"2024-09-21T23:18:38.791983+0000","last_unstale":"2024-09-21T23:18:51.028231+0000","last_undegraded":"2024-09-21T23:18:51.028231+0000","last_fullsized":"2024-09-21T23:18:51.028231+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:55.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"178'938","reported_seq":8298,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028293+0000","last_change":"2024-09-21T23:18:39.806128+0000","last_active":"2024-09-21T23:18:51.028293+0000","last_peered":"2024-09-21T23:18:51.028293+0000","last_clean":"2024-09-21T23:18:51.028293+0000","last_became_active":"2024-09-21T23:18:39.801495+0000","last_became_peered":"2024-09-21T23:18:39.801495+0000","last_unstale":"2024-09-21T23:18:51.028293+0000","last_undegraded":"2024-09-21T23:18:51.028293+0000","last_fullsized":"2024-09-21T23:18:51.028293+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:08.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074401700000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028633+0000","last_change":"2024-09-21T23:18:33.987293+0000","last_active":"2024-09-21T23:18:51.028633+0000","last_peered":"2024-09-21T23:18:51.028633+0000","last_clean":"2024-09-21T23:18:51.028633+0000","last_became_active":"2024-09-21T23:18:33.986262+0000","last_became_peered":"2024-09-21T23:18:33.986262+0000","last_unstale":"2024-09-21T23:18:51.028633+0000","last_undegraded":"2024-09-21T23:18:51.028633+0000","last_fullsized":"2024-09-21T23:18:51.028633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:53.298589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20288,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790013+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:51.790013+0000","last_peered":"2024-09-21T23:18:51.790013+0000","last_clean":"2024-09-21T23:18:51.790013+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:51.790013+0000","last_undegraded":"2024-09-21T23:18:51.790013+0000","last_fullsized":"2024-09-21T23:18:51.790013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029619+0000","last_change":"2024-09-21T23:18:33.986624+0000","last_active":"2024-09-21T23:18:51.029619+0000","last_peered":"2024-09-21T23:18:51.029619+0000","last_clean":"2024-09-21T23:18:51.029619+0000","last_became_active":"2024-09-21T23:18:33.986227+0000","last_became_peered":"2024-09-21T23:18:33.986227+0000","last_unstale":"2024-09-21T23:18:51.029619+0000","last_undegraded":"2024-09-21T23:18:51.029619+0000","last_fullsized":"2024-09-21T23:18:51.029619+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:46.791389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8599,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790716+0000","last_change":"2024-09-21T23:18:48.797459+0000","last_active":"2024-09-21T23:18:51.790716+0000","last_peered":"2024-09-21T23:18:51.790716+0000","last_clean":"2024-09-21T23:18:51.790716+0000","last_became_active":"2024-09-21T23:18:48.795281+0000","last_became_peered":"2024-09-21T23:18:48.795281+0000","last_unstale":"2024-09-21T23:18:51.790716+0000","last_undegraded":"2024-09-21T23:18:51.790716+0000","last_fullsized":"2024-09-21T23:18:51.790716+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:12:45.760380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244768,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028468+0000","last_change":"2024-09-21T23:18:33.982766+0000","last_active":"2024-09-21T23:18:51.028468+0000","last_peered":"2024-09-21T23:18:51.028468+0000","last_clean":"2024-09-21T23:18:51.028468+0000","last_became_active":"2024-09-21T23:18:33.981926+0000","last_became_peered":"2024-09-21T23:18:33.981926+0000","last_unstale":"2024-09-21T23:18:51.028468+0000","last_undegraded":"2024-09-21T23:18:51.028468+0000","last_fullsized":"2024-09-21T23:18:51.028468+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:06.668878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":3308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029990+0000","last_change":"2024-09-21T23:18:47.796589+0000","last_active":"2024-09-21T23:18:51.029990+0000","last_peered":"2024-09-21T23:18:51.029990+0000","last_clean":"2024-09-21T23:18:51.029990+0000","last_became_active":"2024-09-21T23:18:47.794405+0000","last_became_peered":"2024-09-21T23:18:47.794405+0000","last_unstale":"2024-09-21T23:18:51.029990+0000","last_undegraded":"2024-09-21T23:18:51.029990+0000","last_fullsized":"2024-09-21T23:18:51.029990+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:07:32.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073231999999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":269,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029959+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:51.029959+0000","last_peered":"2024-09-21T23:18:51.029959+0000","last_clean":"2024-09-21T23:18:51.029959+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:51.029959+0000","last_undegraded":"2024-09-21T23:18:51.029959+0000","last_fullsized":"2024-09-21T23:18:51.029959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":11175,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788244+0000","last_change":"2024-09-21T23:18:42.790134+0000","last_active":"2024-09-21T23:18:51.788244+0000","last_peered":"2024-09-21T23:18:51.788244+0000","last_clean":"2024-09-21T23:18:51.788244+0000","last_became_active":"2024-09-21T23:18:42.787666+0000","last_became_peered":"2024-09-21T23:18:42.787666+0000","last_unstale":"2024-09-21T23:18:51.788244+0000","last_undegraded":"2024-09-21T23:18:51.788244+0000","last_fullsized":"2024-09-21T23:18:51.788244+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:20.552688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069917950000000003,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'792","reported_seq":7120,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787876+0000","last_change":"2024-09-21T23:18:29.953189+0000","last_active":"2024-09-21T23:18:51.787876+0000","last_peered":"2024-09-21T23:18:51.787876+0000","last_clean":"2024-09-21T23:18:51.787876+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:51.787876+0000","last_undegraded":"2024-09-21T23:18:51.787876+0000","last_fullsized":"2024-09-21T23:18:51.787876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:20.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046990039999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8412,"num_read_kb":29133,"num_write":771,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":404,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787751+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:51.787751+0000","last_peered":"2024-09-21T23:18:51.787751+0000","last_clean":"2024-09-21T23:18:51.787751+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:51.787751+0000","last_undegraded":"2024-09-21T23:18:51.787751+0000","last_fullsized":"2024-09-21T23:18:51.787751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6682,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029845+0000","last_change":"2024-09-21T23:18:47.800836+0000","last_active":"2024-09-21T23:18:51.029845+0000","last_peered":"2024-09-21T23:18:51.029845+0000","last_clean":"2024-09-21T23:18:51.029845+0000","last_became_active":"2024-09-21T23:18:47.798894+0000","last_became_peered":"2024-09-21T23:18:47.798894+0000","last_unstale":"2024-09-21T23:18:51.029845+0000","last_undegraded":"2024-09-21T23:18:51.029845+0000","last_fullsized":"2024-09-21T23:18:51.029845+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:19.199753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015079399999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790577+0000","last_change":"2024-09-21T23:18:42.788361+0000","last_active":"2024-09-21T23:18:51.790577+0000","last_peered":"2024-09-21T23:18:51.790577+0000","last_clean":"2024-09-21T23:18:51.790577+0000","last_became_active":"2024-09-21T23:18:42.788015+0000","last_became_peered":"2024-09-21T23:18:42.788015+0000","last_unstale":"2024-09-21T23:18:51.790577+0000","last_undegraded":"2024-09-21T23:18:51.790577+0000","last_fullsized":"2024-09-21T23:18:51.790577+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:44.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4792,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790731+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:51.790731+0000","last_peered":"2024-09-21T23:18:51.790731+0000","last_clean":"2024-09-21T23:18:51.790731+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:51.790731+0000","last_undegraded":"2024-09-21T23:18:51.790731+0000","last_fullsized":"2024-09-21T23:18:51.790731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027766+0000","last_change":"2024-09-21T23:18:33.978285+0000","last_active":"2024-09-21T23:18:51.027766+0000","last_peered":"2024-09-21T23:18:51.027766+0000","last_clean":"2024-09-21T23:18:51.027766+0000","last_became_active":"2024-09-21T23:18:33.977449+0000","last_became_peered":"2024-09-21T23:18:33.977449+0000","last_unstale":"2024-09-21T23:18:51.027766+0000","last_undegraded":"2024-09-21T23:18:51.027766+0000","last_fullsized":"2024-09-21T23:18:51.027766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:03.716381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":17031,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029958+0000","last_change":"2024-09-21T23:18:30.017566+0000","last_active":"2024-09-21T23:18:51.029958+0000","last_peered":"2024-09-21T23:18:51.029958+0000","last_clean":"2024-09-21T23:18:51.029958+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:51.029958+0000","last_undegraded":"2024-09-21T23:18:51.029958+0000","last_fullsized":"2024-09-21T23:18:51.029958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:56.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020263540000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788003+0000","last_change":"2024-09-21T23:18:42.793227+0000","last_active":"2024-09-21T23:18:51.788003+0000","last_peered":"2024-09-21T23:18:51.788003+0000","last_clean":"2024-09-21T23:18:51.788003+0000","last_became_active":"2024-09-21T23:18:42.788526+0000","last_became_peered":"2024-09-21T23:18:42.788526+0000","last_unstale":"2024-09-21T23:18:51.788003+0000","last_undegraded":"2024-09-21T23:18:51.788003+0000","last_fullsized":"2024-09-21T23:18:51.788003+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:53.455486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":483,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027862+0000","last_change":"2024-09-21T23:18:36.778564+0000","last_active":"2024-09-21T23:18:51.027862+0000","last_peered":"2024-09-21T23:18:51.027862+0000","last_clean":"2024-09-21T23:18:51.027862+0000","last_became_active":"2024-09-21T23:18:36.777596+0000","last_became_peered":"2024-09-21T23:18:36.777596+0000","last_unstale":"2024-09-21T23:18:51.027862+0000","last_undegraded":"2024-09-21T23:18:51.027862+0000","last_fullsized":"2024-09-21T23:18:51.027862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:32.921245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12928,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790372+0000","last_change":"2024-09-21T23:18:47.797129+0000","last_active":"2024-09-21T23:18:51.790372+0000","last_peered":"2024-09-21T23:18:51.790372+0000","last_clean":"2024-09-21T23:18:51.790372+0000","last_became_active":"2024-09-21T23:18:47.793978+0000","last_became_peered":"2024-09-21T23:18:47.793978+0000","last_unstale":"2024-09-21T23:18:51.790372+0000","last_undegraded":"2024-09-21T23:18:51.790372+0000","last_fullsized":"2024-09-21T23:18:51.790372+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:42.993883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011535569999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"178'592","reported_seq":5301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787636+0000","last_change":"2024-09-21T23:18:46.790171+0000","last_active":"2024-09-21T23:18:51.787636+0000","last_peered":"2024-09-21T23:18:51.787636+0000","last_clean":"2024-09-21T23:18:51.787636+0000","last_became_active":"2024-09-21T23:18:46.787797+0000","last_became_peered":"2024-09-21T23:18:46.787797+0000","last_unstale":"2024-09-21T23:18:51.787636+0000","last_undegraded":"2024-09-21T23:18:51.787636+0000","last_fullsized":"2024-09-21T23:18:51.787636+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:43:38.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085867570000000004,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"178'354","reported_seq":4260,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027978+0000","last_change":"2024-09-21T23:18:38.801696+0000","last_active":"2024-09-21T23:18:51.027978+0000","last_peered":"2024-09-21T23:18:51.027978+0000","last_clean":"2024-09-21T23:18:51.027978+0000","last_became_active":"2024-09-21T23:18:38.795761+0000","last_became_peered":"2024-09-21T23:18:38.795761+0000","last_unstale":"2024-09-21T23:18:51.027978+0000","last_undegraded":"2024-09-21T23:18:51.027978+0000","last_fullsized":"2024-09-21T23:18:51.027978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:33.754942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"177'791","reported_seq":8669,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791035+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:51.791035+0000","last_peered":"2024-09-21T23:18:51.791035+0000","last_clean":"2024-09-21T23:18:51.791035+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:51.791035+0000","last_undegraded":"2024-09-21T23:18:51.791035+0000","last_fullsized":"2024-09-21T23:18:51.791035+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"179'686","reported_seq":6156,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029573+0000","last_change":"2024-09-21T23:18:42.790726+0000","last_active":"2024-09-21T23:18:51.029573+0000","last_peered":"2024-09-21T23:18:51.029573+0000","last_clean":"2024-09-21T23:18:51.029573+0000","last_became_active":"2024-09-21T23:18:42.788002+0000","last_became_peered":"2024-09-21T23:18:42.788002+0000","last_unstale":"2024-09-21T23:18:51.029573+0000","last_undegraded":"2024-09-21T23:18:51.029573+0000","last_fullsized":"2024-09-21T23:18:51.029573+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:04:27.488128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062598719999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7976,"num_read_kb":18238,"num_write":516,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"177'475","reported_seq":5551,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028181+0000","last_change":"2024-09-21T23:18:46.790120+0000","last_active":"2024-09-21T23:18:51.028181+0000","last_peered":"2024-09-21T23:18:51.028181+0000","last_clean":"2024-09-21T23:18:51.028181+0000","last_became_active":"2024-09-21T23:18:46.786927+0000","last_became_peered":"2024-09-21T23:18:46.786927+0000","last_unstale":"2024-09-21T23:18:51.028181+0000","last_undegraded":"2024-09-21T23:18:51.028181+0000","last_fullsized":"2024-09-21T23:18:51.028181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:24:25.703682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003514428,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"173'472","reported_seq":4619,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788417+0000","last_change":"2024-09-21T23:18:29.962695+0000","last_active":"2024-09-21T23:18:51.788417+0000","last_peered":"2024-09-21T23:18:51.788417+0000","last_clean":"2024-09-21T23:18:51.788417+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:51.788417+0000","last_undegraded":"2024-09-21T23:18:51.788417+0000","last_fullsized":"2024-09-21T23:18:51.788417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:37.106862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073184640000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'479","reported_seq":6832,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788736+0000","last_change":"2024-09-21T23:18:36.787585+0000","last_active":"2024-09-21T23:18:51.788736+0000","last_peered":"2024-09-21T23:18:51.788736+0000","last_clean":"2024-09-21T23:18:51.788736+0000","last_became_active":"2024-09-21T23:18:36.784584+0000","last_became_peered":"2024-09-21T23:18:36.784584+0000","last_unstale":"2024-09-21T23:18:51.788736+0000","last_undegraded":"2024-09-21T23:18:51.788736+0000","last_fullsized":"2024-09-21T23:18:51.788736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:10:34.977998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080862480000000007,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"178'776","reported_seq":6199,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788078+0000","last_change":"2024-09-21T23:18:29.956286+0000","last_active":"2024-09-21T23:18:51.788078+0000","last_peered":"2024-09-21T23:18:51.788078+0000","last_clean":"2024-09-21T23:18:51.788078+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:51.788078+0000","last_undegraded":"2024-09-21T23:18:51.788078+0000","last_fullsized":"2024-09-21T23:18:51.788078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:23:05.964943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028297349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"178'539","reported_seq":5572,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027938+0000","last_change":"2024-09-21T23:18:40.806974+0000","last_active":"2024-09-21T23:18:51.027938+0000","last_peered":"2024-09-21T23:18:51.027938+0000","last_clean":"2024-09-21T23:18:51.027938+0000","last_became_active":"2024-09-21T23:18:40.802654+0000","last_became_peered":"2024-09-21T23:18:40.802654+0000","last_unstale":"2024-09-21T23:18:51.027938+0000","last_undegraded":"2024-09-21T23:18:51.027938+0000","last_fullsized":"2024-09-21T23:18:51.027938+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:51.646949+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029604689999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"177'807","reported_seq":22299,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790855+0000","last_change":"2024-09-21T23:18:40.819974+0000","last_active":"2024-09-21T23:18:51.790855+0000","last_peered":"2024-09-21T23:18:51.790855+0000","last_clean":"2024-09-21T23:18:51.790855+0000","last_became_active":"2024-09-21T23:18:40.816843+0000","last_became_peered":"2024-09-21T23:18:40.816843+0000","last_unstale":"2024-09-21T23:18:51.790855+0000","last_undegraded":"2024-09-21T23:18:51.790855+0000","last_fullsized":"2024-09-21T23:18:51.790855+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:12.452126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"179'484","reported_seq":9830,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028099+0000","last_change":"2024-09-21T23:18:38.798246+0000","last_active":"2024-09-21T23:18:51.028099+0000","last_peered":"2024-09-21T23:18:51.028099+0000","last_clean":"2024-09-21T23:18:51.028099+0000","last_became_active":"2024-09-21T23:18:38.791632+0000","last_became_peered":"2024-09-21T23:18:38.791632+0000","last_unstale":"2024-09-21T23:18:51.028099+0000","last_undegraded":"2024-09-21T23:18:51.028099+0000","last_fullsized":"2024-09-21T23:18:51.028099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:30.701459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025997469999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":527,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":330,"num_read":313855,"num_read_kb":1079349,"num_write":21231,"num_write_kb":208209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4344,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20259,"ondisk_log_size":20259,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51596,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320244,"statfs":{"total":96636764160,"available":96583929856,"internally_reserved":0,"allocated":2408448,"data_stored":1008181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55736,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316104,"statfs":{"total":96636764160,"available":96579690496,"internally_reserved":0,"allocated":2396160,"data_stored":1003182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607607,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42436,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329404,"statfs":{"total":96636764160,"available":96593309696,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51588,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320252,"statfs":{"total":96636764160,"available":96583938048,"internally_reserved":0,"allocated":2404352,"data_stored":1008162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607608,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77968,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293872,"statfs":{"total":96636764160,"available":96556924928,"internally_reserved":0,"allocated":2412544,"data_stored":1008200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005092,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61628,"kb_used_data":2916,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310212,"statfs":{"total":96636764160,"available":96573657088,"internally_reserved":0,"allocated":2985984,"data_stored":1593531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:18:55.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T23:18:56.270 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:18:56.270 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":200,"fsid":"309803bf-3365-47ac-8bf6-365125beceb1","created":"2024-09-21T23:14:32.028556+0000","modified":"2024-09-21T23:18:51.024572+0000","last_up_change":"2024-09-21T23:16:02.149456+0000","last_in_change":"2024-09-21T23:18:38.779988+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T23:14:44.926374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T23:14:49.598603+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T23:14:59.367341+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":42,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a24df615-4443-4f27-888f-d1e4e7affbb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":29,"up_from":61,"up_thru":195,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6817","nonce":1028371170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6819","nonce":1028371170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6823","nonce":1028371170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1028371170},{"type":"v1","addr":"172.21.15.53:6821","nonce":1028371170}]},"public_addr":"172.21.15.53:6817/1028371170","cluster_addr":"172.21.15.53:6819/1028371170","heartbeat_back_addr":"172.21.15.53:6823/1028371170","heartbeat_front_addr":"172.21.15.53:6821/1028371170","state":["exists","up"]},{"osd":1,"uuid":"d6765c2a-98b2-4a2b-8d8c-f3963575eb64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6809","nonce":2712042978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6811","nonce":2712042978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6815","nonce":2712042978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2712042978},{"type":"v1","addr":"172.21.15.53:6813","nonce":2712042978}]},"public_addr":"172.21.15.53:6809/2712042978","cluster_addr":"172.21.15.53:6811/2712042978","heartbeat_back_addr":"172.21.15.53:6815/2712042978","heartbeat_front_addr":"172.21.15.53:6813/2712042978","state":["exists","up"]},{"osd":2,"uuid":"9753272a-58a9-4366-87d7-550c49134023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6801","nonce":1166517858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6803","nonce":1166517858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6807","nonce":1166517858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1166517858},{"type":"v1","addr":"172.21.15.53:6805","nonce":1166517858}]},"public_addr":"172.21.15.53:6801/1166517858","cluster_addr":"172.21.15.53:6803/1166517858","heartbeat_back_addr":"172.21.15.53:6807/1166517858","heartbeat_front_addr":"172.21.15.53:6805/1166517858","state":["exists","up"]},{"osd":3,"uuid":"24d987f9-f2cd-4ccd-9dc8-fea64467d92d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6811","nonce":1039392120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6816","nonce":1039392120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6823","nonce":1039392120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1039392120},{"type":"v1","addr":"172.21.15.144:6821","nonce":1039392120}]},"public_addr":"172.21.15.144:6811/1039392120","cluster_addr":"172.21.15.144:6816/1039392120","heartbeat_back_addr":"172.21.15.144:6823/1039392120","heartbeat_front_addr":"172.21.15.144:6821/1039392120","state":["exists","up"]},{"osd":4,"uuid":"c1c30a49-7f26-4264-a1a9-160dc552953d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6810","nonce":2764095640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6813","nonce":2764095640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6820","nonce":2764095640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6815","nonce":2764095640},{"type":"v1","addr":"172.21.15.144:6817","nonce":2764095640}]},"public_addr":"172.21.15.144:6810/2764095640","cluster_addr":"172.21.15.144:6813/2764095640","heartbeat_back_addr":"172.21.15.144:6820/2764095640","heartbeat_front_addr":"172.21.15.144:6817/2764095640","state":["exists","up"]},{"osd":5,"uuid":"f24e9341-be4e-4be2-b180-7ebd3f705e9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6801","nonce":1732973431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6803","nonce":1732973431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6807","nonce":1732973431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1732973431},{"type":"v1","addr":"172.21.15.144:6805","nonce":1732973431}]},"public_addr":"172.21.15.144:6801/1732973431","cluster_addr":"172.21.15.144:6803/1732973431","heartbeat_back_addr":"172.21.15.144:6807/1732973431","heartbeat_front_addr":"172.21.15.144:6805/1732973431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.468749+0000","dead_epoch":31},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.044837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.310970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.241402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.877818+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T23:14:41.434052+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.b","osds":[0,4]},{"pgid":"3.c","osds":[1,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T23:18:57.290 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T23:18:57.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:18:57.452 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:18:57.452 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:57.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T23:18:57.759 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T23:18:57.778 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T23:18:57.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:18:57.926 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:57.926 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:18:57.926 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:57.942 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T23:18:58.231 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T23:18:58.250 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T23:18:58.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:18:58.394 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:18:58.395 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:58.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T23:18:58.700 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T23:18:58.720 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T23:18:58.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:18:58.873 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:58.873 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:18:58.873 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:58.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T23:18:59.182 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T23:18:59.201 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T23:18:59.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:18:59.353 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:59.353 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:18:59.353 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:59.369 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T23:18:59.659 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T23:18:59.676 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T23:18:59.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T23:18:59.829 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T23:18:59.829 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T23:18:59.829 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T23:18:59.845 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T23:19:00.141 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T23:19:00.160 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:19:00.472 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:19:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:19:00.490 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-21T23:18:58.911799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":338,"num_object_clones":0,"num_object_copies":676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":330,"num_read":313971,"num_read_kb":1079449,"num_write":21392,"num_write_kb":210627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4354,"num_bytes_recovered":24942653,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20379,"ondisk_log_size":20379,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340984,"kb_used_data":15264,"kb_used_omap":13,"kb_used_meta":325554,"kb_avail":565890056,"statfs":{"total":579820584960,"available":579471417344,"internally_reserved":0,"allocated":15630336,"data_stored":7229784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13824,"internal_metadata":333367808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.897214"},"pg_stats":[{"pgid":"3.1f","version":"179'567","reported_seq":13668,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790048+0000","last_change":"2024-09-21T23:18:40.802504+0000","last_active":"2024-09-21T23:18:51.790048+0000","last_peered":"2024-09-21T23:18:51.790048+0000","last_clean":"2024-09-21T23:18:51.790048+0000","last_became_active":"2024-09-21T23:18:40.801017+0000","last_became_peered":"2024-09-21T23:18:40.801017+0000","last_unstale":"2024-09-21T23:18:51.790048+0000","last_undegraded":"2024-09-21T23:18:51.790048+0000","last_fullsized":"2024-09-21T23:18:51.790048+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:02.839217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'484","reported_seq":5924,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790724+0000","last_change":"2024-09-21T23:18:46.792370+0000","last_active":"2024-09-21T23:18:51.790724+0000","last_peered":"2024-09-21T23:18:51.790724+0000","last_clean":"2024-09-21T23:18:51.790724+0000","last_became_active":"2024-09-21T23:18:46.789102+0000","last_became_peered":"2024-09-21T23:18:46.789102+0000","last_unstale":"2024-09-21T23:18:51.790724+0000","last_undegraded":"2024-09-21T23:18:51.790724+0000","last_fullsized":"2024-09-21T23:18:51.790724+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:39.532998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020600919999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"177'597","reported_seq":8089,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788065+0000","last_change":"2024-09-21T23:18:29.959111+0000","last_active":"2024-09-21T23:18:51.788065+0000","last_peered":"2024-09-21T23:18:51.788065+0000","last_clean":"2024-09-21T23:18:51.788065+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:18:51.788065+0000","last_undegraded":"2024-09-21T23:18:51.788065+0000","last_fullsized":"2024-09-21T23:18:51.788065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:19.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053022649999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"179'1015","reported_seq":13065,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787713+0000","last_change":"2024-09-21T23:18:29.959245+0000","last_active":"2024-09-21T23:18:51.787713+0000","last_peered":"2024-09-21T23:18:51.787713+0000","last_clean":"2024-09-21T23:18:51.787713+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:18:51.787713+0000","last_undegraded":"2024-09-21T23:18:51.787713+0000","last_fullsized":"2024-09-21T23:18:51.787713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:45.242522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048112889999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17738,"num_read_kb":71148,"num_write":1049,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'490","reported_seq":5328,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791127+0000","last_change":"2024-09-21T23:18:29.950817+0000","last_active":"2024-09-21T23:18:51.791127+0000","last_peered":"2024-09-21T23:18:51.791127+0000","last_clean":"2024-09-21T23:18:51.791127+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:18:51.791127+0000","last_undegraded":"2024-09-21T23:18:51.791127+0000","last_fullsized":"2024-09-21T23:18:51.791127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:35.080505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"178'696","reported_seq":10529,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787725+0000","last_change":"2024-09-21T23:18:29.954505+0000","last_active":"2024-09-21T23:18:51.787725+0000","last_peered":"2024-09-21T23:18:51.787725+0000","last_clean":"2024-09-21T23:18:51.787725+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:18:51.787725+0000","last_undegraded":"2024-09-21T23:18:51.787725+0000","last_fullsized":"2024-09-21T23:18:51.787725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:44:47.554016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059155759999999996,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12157,"num_read_kb":26818,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'411","reported_seq":5443,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029539+0000","last_change":"2024-09-21T23:18:30.016900+0000","last_active":"2024-09-21T23:18:51.029539+0000","last_peered":"2024-09-21T23:18:51.029539+0000","last_clean":"2024-09-21T23:18:51.029539+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:51.029539+0000","last_undegraded":"2024-09-21T23:18:51.029539+0000","last_fullsized":"2024-09-21T23:18:51.029539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:36:28.556199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016355880000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'672","reported_seq":18660,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790494+0000","last_change":"2024-09-21T23:18:47.800021+0000","last_active":"2024-09-21T23:18:51.790494+0000","last_peered":"2024-09-21T23:18:51.790494+0000","last_clean":"2024-09-21T23:18:51.790494+0000","last_became_active":"2024-09-21T23:18:47.797088+0000","last_became_peered":"2024-09-21T23:18:47.797088+0000","last_unstale":"2024-09-21T23:18:51.790494+0000","last_undegraded":"2024-09-21T23:18:51.790494+0000","last_fullsized":"2024-09-21T23:18:51.790494+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:31.792480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"178'416","reported_seq":13674,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790017+0000","last_change":"2024-09-21T23:18:29.948902+0000","last_active":"2024-09-21T23:18:51.790017+0000","last_peered":"2024-09-21T23:18:51.790017+0000","last_clean":"2024-09-21T23:18:51.790017+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:51.790017+0000","last_undegraded":"2024-09-21T23:18:51.790017+0000","last_fullsized":"2024-09-21T23:18:51.790017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:03.386039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"178'581","reported_seq":6245,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788309+0000","last_change":"2024-09-21T23:18:39.802796+0000","last_active":"2024-09-21T23:18:51.788309+0000","last_peered":"2024-09-21T23:18:51.788309+0000","last_clean":"2024-09-21T23:18:51.788309+0000","last_became_active":"2024-09-21T23:18:39.798603+0000","last_became_peered":"2024-09-21T23:18:39.798603+0000","last_unstale":"2024-09-21T23:18:51.788309+0000","last_undegraded":"2024-09-21T23:18:51.788309+0000","last_fullsized":"2024-09-21T23:18:51.788309+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:00:20.739193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061281979999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"177'349","reported_seq":4291,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028231+0000","last_change":"2024-09-21T23:18:38.796650+0000","last_active":"2024-09-21T23:18:51.028231+0000","last_peered":"2024-09-21T23:18:51.028231+0000","last_clean":"2024-09-21T23:18:51.028231+0000","last_became_active":"2024-09-21T23:18:38.791983+0000","last_became_peered":"2024-09-21T23:18:38.791983+0000","last_unstale":"2024-09-21T23:18:51.028231+0000","last_undegraded":"2024-09-21T23:18:51.028231+0000","last_fullsized":"2024-09-21T23:18:51.028231+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:55.929059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"178'938","reported_seq":8298,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028293+0000","last_change":"2024-09-21T23:18:39.806128+0000","last_active":"2024-09-21T23:18:51.028293+0000","last_peered":"2024-09-21T23:18:51.028293+0000","last_clean":"2024-09-21T23:18:51.028293+0000","last_became_active":"2024-09-21T23:18:39.801495+0000","last_became_peered":"2024-09-21T23:18:39.801495+0000","last_unstale":"2024-09-21T23:18:51.028293+0000","last_undegraded":"2024-09-21T23:18:51.028293+0000","last_fullsized":"2024-09-21T23:18:51.028293+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:35:08.682437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074401700000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028633+0000","last_change":"2024-09-21T23:18:33.987293+0000","last_active":"2024-09-21T23:18:51.028633+0000","last_peered":"2024-09-21T23:18:51.028633+0000","last_clean":"2024-09-21T23:18:51.028633+0000","last_became_active":"2024-09-21T23:18:33.986262+0000","last_became_peered":"2024-09-21T23:18:33.986262+0000","last_unstale":"2024-09-21T23:18:51.028633+0000","last_undegraded":"2024-09-21T23:18:51.028633+0000","last_fullsized":"2024-09-21T23:18:51.028633+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:53.298589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"179'767","reported_seq":20288,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790013+0000","last_change":"2024-09-21T23:18:29.947728+0000","last_active":"2024-09-21T23:18:51.790013+0000","last_peered":"2024-09-21T23:18:51.790013+0000","last_clean":"2024-09-21T23:18:51.790013+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:18:51.790013+0000","last_undegraded":"2024-09-21T23:18:51.790013+0000","last_fullsized":"2024-09-21T23:18:51.790013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:16.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029619+0000","last_change":"2024-09-21T23:18:33.986624+0000","last_active":"2024-09-21T23:18:51.029619+0000","last_peered":"2024-09-21T23:18:51.029619+0000","last_clean":"2024-09-21T23:18:51.029619+0000","last_became_active":"2024-09-21T23:18:33.986227+0000","last_became_peered":"2024-09-21T23:18:33.986227+0000","last_unstale":"2024-09-21T23:18:51.029619+0000","last_undegraded":"2024-09-21T23:18:51.029619+0000","last_fullsized":"2024-09-21T23:18:51.029619+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:42:46.791389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'1656","reported_seq":8599,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790716+0000","last_change":"2024-09-21T23:18:48.797459+0000","last_active":"2024-09-21T23:18:51.790716+0000","last_peered":"2024-09-21T23:18:51.790716+0000","last_clean":"2024-09-21T23:18:51.790716+0000","last_became_active":"2024-09-21T23:18:48.795281+0000","last_became_peered":"2024-09-21T23:18:48.795281+0000","last_unstale":"2024-09-21T23:18:51.790716+0000","last_undegraded":"2024-09-21T23:18:51.790716+0000","last_fullsized":"2024-09-21T23:18:51.790716+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":1656,"log_dups_size":0,"ondisk_log_size":1656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:12:45.760380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244768,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028468+0000","last_change":"2024-09-21T23:18:33.982766+0000","last_active":"2024-09-21T23:18:51.028468+0000","last_peered":"2024-09-21T23:18:51.028468+0000","last_clean":"2024-09-21T23:18:51.028468+0000","last_became_active":"2024-09-21T23:18:33.981926+0000","last_became_peered":"2024-09-21T23:18:33.981926+0000","last_unstale":"2024-09-21T23:18:51.028468+0000","last_undegraded":"2024-09-21T23:18:51.028468+0000","last_fullsized":"2024-09-21T23:18:51.028468+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:24:06.668878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"175'463","reported_seq":3308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029990+0000","last_change":"2024-09-21T23:18:47.796589+0000","last_active":"2024-09-21T23:18:51.029990+0000","last_peered":"2024-09-21T23:18:51.029990+0000","last_clean":"2024-09-21T23:18:51.029990+0000","last_became_active":"2024-09-21T23:18:47.794405+0000","last_became_peered":"2024-09-21T23:18:47.794405+0000","last_unstale":"2024-09-21T23:18:51.029990+0000","last_undegraded":"2024-09-21T23:18:51.029990+0000","last_fullsized":"2024-09-21T23:18:51.029990+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:07:32.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073231999999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":269,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029959+0000","last_change":"2024-09-21T23:16:15.267603+0000","last_active":"2024-09-21T23:18:51.029959+0000","last_peered":"2024-09-21T23:18:51.029959+0000","last_clean":"2024-09-21T23:18:51.029959+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:51.029959+0000","last_undegraded":"2024-09-21T23:18:51.029959+0000","last_fullsized":"2024-09-21T23:18:51.029959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:58.214423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'559","reported_seq":11175,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788244+0000","last_change":"2024-09-21T23:18:42.790134+0000","last_active":"2024-09-21T23:18:51.788244+0000","last_peered":"2024-09-21T23:18:51.788244+0000","last_clean":"2024-09-21T23:18:51.788244+0000","last_became_active":"2024-09-21T23:18:42.787666+0000","last_became_peered":"2024-09-21T23:18:42.787666+0000","last_unstale":"2024-09-21T23:18:51.788244+0000","last_undegraded":"2024-09-21T23:18:51.788244+0000","last_fullsized":"2024-09-21T23:18:51.788244+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:20.552688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069917950000000003,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'792","reported_seq":7120,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787876+0000","last_change":"2024-09-21T23:18:29.953189+0000","last_active":"2024-09-21T23:18:51.787876+0000","last_peered":"2024-09-21T23:18:51.787876+0000","last_clean":"2024-09-21T23:18:51.787876+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:18:51.787876+0000","last_undegraded":"2024-09-21T23:18:51.787876+0000","last_fullsized":"2024-09-21T23:18:51.787876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:38:20.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046990039999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8412,"num_read_kb":29133,"num_write":771,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":404,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787751+0000","last_change":"2024-09-21T23:16:15.270260+0000","last_active":"2024-09-21T23:18:51.787751+0000","last_peered":"2024-09-21T23:18:51.787751+0000","last_clean":"2024-09-21T23:18:51.787751+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:18:51.787751+0000","last_undegraded":"2024-09-21T23:18:51.787751+0000","last_fullsized":"2024-09-21T23:18:51.787751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:38:36.586732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"178'371","reported_seq":6682,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029845+0000","last_change":"2024-09-21T23:18:47.800836+0000","last_active":"2024-09-21T23:18:51.029845+0000","last_peered":"2024-09-21T23:18:51.029845+0000","last_clean":"2024-09-21T23:18:51.029845+0000","last_became_active":"2024-09-21T23:18:47.798894+0000","last_became_peered":"2024-09-21T23:18:47.798894+0000","last_unstale":"2024-09-21T23:18:51.029845+0000","last_undegraded":"2024-09-21T23:18:51.029845+0000","last_fullsized":"2024-09-21T23:18:51.029845+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:49:19.199753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015079399999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790577+0000","last_change":"2024-09-21T23:18:42.788361+0000","last_active":"2024-09-21T23:18:51.790577+0000","last_peered":"2024-09-21T23:18:51.790577+0000","last_clean":"2024-09-21T23:18:51.790577+0000","last_became_active":"2024-09-21T23:18:42.788015+0000","last_became_peered":"2024-09-21T23:18:42.788015+0000","last_unstale":"2024-09-21T23:18:51.790577+0000","last_undegraded":"2024-09-21T23:18:51.790577+0000","last_fullsized":"2024-09-21T23:18:51.790577+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:26:44.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'425","reported_seq":4792,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790731+0000","last_change":"2024-09-21T23:18:29.945885+0000","last_active":"2024-09-21T23:18:51.790731+0000","last_peered":"2024-09-21T23:18:51.790731+0000","last_clean":"2024-09-21T23:18:51.790731+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:51.790731+0000","last_undegraded":"2024-09-21T23:18:51.790731+0000","last_fullsized":"2024-09-21T23:18:51.790731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:58.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027766+0000","last_change":"2024-09-21T23:18:33.978285+0000","last_active":"2024-09-21T23:18:51.027766+0000","last_peered":"2024-09-21T23:18:51.027766+0000","last_clean":"2024-09-21T23:18:51.027766+0000","last_became_active":"2024-09-21T23:18:33.977449+0000","last_became_peered":"2024-09-21T23:18:33.977449+0000","last_unstale":"2024-09-21T23:18:51.027766+0000","last_undegraded":"2024-09-21T23:18:51.027766+0000","last_fullsized":"2024-09-21T23:18:51.027766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:03.716381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"178'651","reported_seq":17031,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029958+0000","last_change":"2024-09-21T23:18:30.017566+0000","last_active":"2024-09-21T23:18:51.029958+0000","last_peered":"2024-09-21T23:18:51.029958+0000","last_clean":"2024-09-21T23:18:51.029958+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:18:51.029958+0000","last_undegraded":"2024-09-21T23:18:51.029958+0000","last_fullsized":"2024-09-21T23:18:51.029958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:46:56.410785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020263540000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":400,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788003+0000","last_change":"2024-09-21T23:18:42.793227+0000","last_active":"2024-09-21T23:18:51.788003+0000","last_peered":"2024-09-21T23:18:51.788003+0000","last_clean":"2024-09-21T23:18:51.788003+0000","last_became_active":"2024-09-21T23:18:42.788526+0000","last_became_peered":"2024-09-21T23:18:42.788526+0000","last_unstale":"2024-09-21T23:18:51.788003+0000","last_undegraded":"2024-09-21T23:18:51.788003+0000","last_fullsized":"2024-09-21T23:18:51.788003+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:50.229607+0000","last_clean_scrub_stamp":"2024-09-21T23:14:50.229607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:53.455486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":483,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027862+0000","last_change":"2024-09-21T23:18:36.778564+0000","last_active":"2024-09-21T23:18:51.027862+0000","last_peered":"2024-09-21T23:18:51.027862+0000","last_clean":"2024-09-21T23:18:51.027862+0000","last_became_active":"2024-09-21T23:18:36.777596+0000","last_became_peered":"2024-09-21T23:18:36.777596+0000","last_unstale":"2024-09-21T23:18:51.027862+0000","last_undegraded":"2024-09-21T23:18:51.027862+0000","last_fullsized":"2024-09-21T23:18:51.027862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:14:45.207027+0000","last_clean_scrub_stamp":"2024-09-21T23:14:45.207027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:52:32.921245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"177'904","reported_seq":12928,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790372+0000","last_change":"2024-09-21T23:18:47.797129+0000","last_active":"2024-09-21T23:18:51.790372+0000","last_peered":"2024-09-21T23:18:51.790372+0000","last_clean":"2024-09-21T23:18:51.790372+0000","last_became_active":"2024-09-21T23:18:47.793978+0000","last_became_peered":"2024-09-21T23:18:47.793978+0000","last_unstale":"2024-09-21T23:18:51.790372+0000","last_undegraded":"2024-09-21T23:18:51.790372+0000","last_fullsized":"2024-09-21T23:18:51.790372+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:42.993883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011535569999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"178'592","reported_seq":5301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.787636+0000","last_change":"2024-09-21T23:18:46.790171+0000","last_active":"2024-09-21T23:18:51.787636+0000","last_peered":"2024-09-21T23:18:51.787636+0000","last_clean":"2024-09-21T23:18:51.787636+0000","last_became_active":"2024-09-21T23:18:46.787797+0000","last_became_peered":"2024-09-21T23:18:46.787797+0000","last_unstale":"2024-09-21T23:18:51.787636+0000","last_undegraded":"2024-09-21T23:18:51.787636+0000","last_fullsized":"2024-09-21T23:18:51.787636+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:43:38.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085867570000000004,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"178'354","reported_seq":4260,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027978+0000","last_change":"2024-09-21T23:18:38.801696+0000","last_active":"2024-09-21T23:18:51.027978+0000","last_peered":"2024-09-21T23:18:51.027978+0000","last_clean":"2024-09-21T23:18:51.027978+0000","last_became_active":"2024-09-21T23:18:38.795761+0000","last_became_peered":"2024-09-21T23:18:38.795761+0000","last_unstale":"2024-09-21T23:18:51.027978+0000","last_undegraded":"2024-09-21T23:18:51.027978+0000","last_fullsized":"2024-09-21T23:18:51.027978+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:33.754942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"177'791","reported_seq":8669,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.791035+0000","last_change":"2024-09-21T23:18:29.950017+0000","last_active":"2024-09-21T23:18:51.791035+0000","last_peered":"2024-09-21T23:18:51.791035+0000","last_clean":"2024-09-21T23:18:51.791035+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:18:51.791035+0000","last_undegraded":"2024-09-21T23:18:51.791035+0000","last_fullsized":"2024-09-21T23:18:51.791035+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:23.638981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"179'686","reported_seq":6156,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.029573+0000","last_change":"2024-09-21T23:18:42.790726+0000","last_active":"2024-09-21T23:18:51.029573+0000","last_peered":"2024-09-21T23:18:51.029573+0000","last_clean":"2024-09-21T23:18:51.029573+0000","last_became_active":"2024-09-21T23:18:42.788002+0000","last_became_peered":"2024-09-21T23:18:42.788002+0000","last_unstale":"2024-09-21T23:18:51.029573+0000","last_undegraded":"2024-09-21T23:18:51.029573+0000","last_fullsized":"2024-09-21T23:18:51.029573+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:04:27.488128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062598719999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7976,"num_read_kb":18238,"num_write":516,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"177'475","reported_seq":5551,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028181+0000","last_change":"2024-09-21T23:18:46.790120+0000","last_active":"2024-09-21T23:18:51.028181+0000","last_peered":"2024-09-21T23:18:51.028181+0000","last_clean":"2024-09-21T23:18:51.028181+0000","last_became_active":"2024-09-21T23:18:46.786927+0000","last_became_peered":"2024-09-21T23:18:46.786927+0000","last_unstale":"2024-09-21T23:18:51.028181+0000","last_undegraded":"2024-09-21T23:18:51.028181+0000","last_fullsized":"2024-09-21T23:18:51.028181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:24:25.703682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003514428,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"173'472","reported_seq":4619,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788417+0000","last_change":"2024-09-21T23:18:29.962695+0000","last_active":"2024-09-21T23:18:51.788417+0000","last_peered":"2024-09-21T23:18:51.788417+0000","last_clean":"2024-09-21T23:18:51.788417+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:18:51.788417+0000","last_undegraded":"2024-09-21T23:18:51.788417+0000","last_fullsized":"2024-09-21T23:18:51.788417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:54:37.106862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073184640000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'479","reported_seq":6832,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788736+0000","last_change":"2024-09-21T23:18:36.787585+0000","last_active":"2024-09-21T23:18:51.788736+0000","last_peered":"2024-09-21T23:18:51.788736+0000","last_clean":"2024-09-21T23:18:51.788736+0000","last_became_active":"2024-09-21T23:18:36.784584+0000","last_became_peered":"2024-09-21T23:18:36.784584+0000","last_unstale":"2024-09-21T23:18:51.788736+0000","last_undegraded":"2024-09-21T23:18:51.788736+0000","last_fullsized":"2024-09-21T23:18:51.788736+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:10:34.977998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080862480000000007,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"178'776","reported_seq":6199,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.788078+0000","last_change":"2024-09-21T23:18:29.956286+0000","last_active":"2024-09-21T23:18:51.788078+0000","last_peered":"2024-09-21T23:18:51.788078+0000","last_clean":"2024-09-21T23:18:51.788078+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:18:51.788078+0000","last_undegraded":"2024-09-21T23:18:51.788078+0000","last_fullsized":"2024-09-21T23:18:51.788078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:23:05.964943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028297349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"178'539","reported_seq":5572,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.027938+0000","last_change":"2024-09-21T23:18:40.806974+0000","last_active":"2024-09-21T23:18:51.027938+0000","last_peered":"2024-09-21T23:18:51.027938+0000","last_clean":"2024-09-21T23:18:51.027938+0000","last_became_active":"2024-09-21T23:18:40.802654+0000","last_became_peered":"2024-09-21T23:18:40.802654+0000","last_unstale":"2024-09-21T23:18:51.027938+0000","last_undegraded":"2024-09-21T23:18:51.027938+0000","last_fullsized":"2024-09-21T23:18:51.027938+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:36:51.646949+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029604689999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"177'807","reported_seq":22299,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.790855+0000","last_change":"2024-09-21T23:18:40.819974+0000","last_active":"2024-09-21T23:18:51.790855+0000","last_peered":"2024-09-21T23:18:51.790855+0000","last_clean":"2024-09-21T23:18:51.790855+0000","last_became_active":"2024-09-21T23:18:40.816843+0000","last_became_peered":"2024-09-21T23:18:40.816843+0000","last_unstale":"2024-09-21T23:18:51.790855+0000","last_undegraded":"2024-09-21T23:18:51.790855+0000","last_fullsized":"2024-09-21T23:18:51.790855+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:12.452126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"179'484","reported_seq":9830,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:51.028099+0000","last_change":"2024-09-21T23:18:38.798246+0000","last_active":"2024-09-21T23:18:51.028099+0000","last_peered":"2024-09-21T23:18:51.028099+0000","last_clean":"2024-09-21T23:18:51.028099+0000","last_became_active":"2024-09-21T23:18:38.791632+0000","last_became_peered":"2024-09-21T23:18:38.791632+0000","last_unstale":"2024-09-21T23:18:51.028099+0000","last_undegraded":"2024-09-21T23:18:51.028099+0000","last_fullsized":"2024-09-21T23:18:51.028099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:15:00.286153+0000","last_clean_scrub_stamp":"2024-09-21T23:15:00.286153+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:30.701459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025997469999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":527,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":330,"num_read":313855,"num_read_kb":1079349,"num_write":21231,"num_write_kb":208209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4344,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20259,"ondisk_log_size":20259,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51596,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320244,"statfs":{"total":96636764160,"available":96583929856,"internally_reserved":0,"allocated":2408448,"data_stored":1008181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55752,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316088,"statfs":{"total":96636764160,"available":96579674112,"internally_reserved":0,"allocated":2408448,"data_stored":1008181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607607,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42436,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329404,"statfs":{"total":96636764160,"available":96593309696,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607607,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51588,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320252,"statfs":{"total":96636764160,"available":96583938048,"internally_reserved":0,"allocated":2404352,"data_stored":1008162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607608,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77968,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293872,"statfs":{"total":96636764160,"available":96556924928,"internally_reserved":0,"allocated":2412544,"data_stored":1008200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005093,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61644,"kb_used_data":2928,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310196,"statfs":{"total":96636764160,"available":96573640704,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:19:00.493 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.494 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.495 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T23:14:50.229607+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=50, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.496 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T23:14:45.207027+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=14, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.497 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.498 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.498 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T23:15:00.286153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=15, tm_sec=0, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=23, tm_min=18, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T23:19:00.498 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T23:19:20.498 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T23:19:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T23:19:20.821 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-21T23:19:20.839 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-21T23:19:18.917995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":338,"num_object_clones":0,"num_object_copies":676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":330,"num_read":313971,"num_read_kb":1079449,"num_write":21392,"num_write_kb":210627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4354,"num_bytes_recovered":24942653,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20717,"ondisk_log_size":20717,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340928,"kb_used_data":15264,"kb_used_omap":13,"kb_used_meta":325554,"kb_avail":565890112,"statfs":{"total":579820584960,"available":579471474688,"internally_reserved":0,"allocated":15630336,"data_stored":7229784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13824,"internal_metadata":333367808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003709"},"pg_stats":[{"pgid":"3.1f","version":"200'582","reported_seq":13692,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:03.363278+0000","last_change":"2024-09-21T23:19:03.363365+0000","last_active":"2024-09-21T23:19:03.363278+0000","last_peered":"2024-09-21T23:19:03.363278+0000","last_clean":"2024-09-21T23:19:03.363278+0000","last_became_active":"2024-09-21T23:18:40.801017+0000","last_became_peered":"2024-09-21T23:18:40.801017+0000","last_unstale":"2024-09-21T23:19:03.363278+0000","last_undegraded":"2024-09-21T23:19:03.363278+0000","last_fullsized":"2024-09-21T23:19:03.363278+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'582","last_scrub_stamp":"2024-09-21T23:19:03.363090+0000","last_deep_scrub":"200'582","last_deep_scrub_stamp":"2024-09-21T23:19:03.363090+0000","last_clean_scrub_stamp":"2024-09-21T23:19:03.363090+0000","objects_scrubbed":15,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:47:58.717921+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0043198059999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":13359,"num_read_kb":21656,"num_write":618,"num_write_kb":3606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":184461,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"200'494","reported_seq":5943,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:59.687940+0000","last_change":"2024-09-21T23:18:59.688027+0000","last_active":"2024-09-21T23:18:59.687940+0000","last_peered":"2024-09-21T23:18:59.687940+0000","last_clean":"2024-09-21T23:18:59.687940+0000","last_became_active":"2024-09-21T23:18:46.789102+0000","last_became_peered":"2024-09-21T23:18:46.789102+0000","last_unstale":"2024-09-21T23:18:59.687940+0000","last_undegraded":"2024-09-21T23:18:59.687940+0000","last_fullsized":"2024-09-21T23:18:59.687940+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'494","last_scrub_stamp":"2024-09-21T23:18:59.687804+0000","last_deep_scrub":"200'494","last_deep_scrub_stamp":"2024-09-21T23:18:59.687804+0000","last_clean_scrub_stamp":"2024-09-21T23:18:59.687804+0000","objects_scrubbed":10,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:43:12.554782+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0020600919999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5573,"num_read_kb":31943,"num_write":417,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":4427368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"200'604","reported_seq":8105,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:01.571209+0000","last_change":"2024-09-21T23:19:01.571367+0000","last_active":"2024-09-21T23:19:01.571209+0000","last_peered":"2024-09-21T23:19:01.571209+0000","last_clean":"2024-09-21T23:19:01.571209+0000","last_became_active":"2024-09-21T23:15:01.313193+0000","last_became_peered":"2024-09-21T23:15:01.313193+0000","last_unstale":"2024-09-21T23:19:01.571209+0000","last_undegraded":"2024-09-21T23:19:01.571209+0000","last_fullsized":"2024-09-21T23:19:01.571209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'604","last_scrub_stamp":"2024-09-21T23:19:01.570989+0000","last_deep_scrub":"200'604","last_deep_scrub_stamp":"2024-09-21T23:19:01.570989+0000","last_clean_scrub_stamp":"2024-09-21T23:19:01.570989+0000","objects_scrubbed":7,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:58:24.906329+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0053022649999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8718,"num_read_kb":41550,"num_write":595,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"200'1025","reported_seq":13084,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:00.591817+0000","last_change":"2024-09-21T23:19:00.591895+0000","last_active":"2024-09-21T23:19:00.591817+0000","last_peered":"2024-09-21T23:19:00.591817+0000","last_clean":"2024-09-21T23:19:00.591817+0000","last_became_active":"2024-09-21T23:15:01.313141+0000","last_became_peered":"2024-09-21T23:15:01.313141+0000","last_unstale":"2024-09-21T23:19:00.591817+0000","last_undegraded":"2024-09-21T23:19:00.591817+0000","last_fullsized":"2024-09-21T23:19:00.591817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'1025","last_scrub_stamp":"2024-09-21T23:19:00.591635+0000","last_deep_scrub":"200'1025","last_deep_scrub_stamp":"2024-09-21T23:19:00.591635+0000","last_clean_scrub_stamp":"2024-09-21T23:19:00.591635+0000","objects_scrubbed":10,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:43:29.728181+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0048112889999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":17738,"num_read_kb":71148,"num_write":1049,"num_write_kb":18799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"200'501","reported_seq":5348,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:00.299279+0000","last_change":"2024-09-21T23:19:00.299318+0000","last_active":"2024-09-21T23:19:00.299279+0000","last_peered":"2024-09-21T23:19:00.299279+0000","last_clean":"2024-09-21T23:19:00.299279+0000","last_became_active":"2024-09-21T23:16:06.219384+0000","last_became_peered":"2024-09-21T23:16:06.219384+0000","last_unstale":"2024-09-21T23:19:00.299279+0000","last_undegraded":"2024-09-21T23:19:00.299279+0000","last_fullsized":"2024-09-21T23:19:00.299279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'501","last_scrub_stamp":"2024-09-21T23:19:00.299201+0000","last_deep_scrub":"200'501","last_deep_scrub_stamp":"2024-09-21T23:19:00.299201+0000","last_clean_scrub_stamp":"2024-09-21T23:19:00.299201+0000","objects_scrubbed":11,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:56:28.387664+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0033796080000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5916,"num_read_kb":22698,"num_write":459,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":92702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"200'710","reported_seq":10552,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:01.449028+0000","last_change":"2024-09-21T23:19:01.449123+0000","last_active":"2024-09-21T23:19:01.449028+0000","last_peered":"2024-09-21T23:19:01.449028+0000","last_clean":"2024-09-21T23:19:01.449028+0000","last_became_active":"2024-09-21T23:16:18.775581+0000","last_became_peered":"2024-09-21T23:16:18.775581+0000","last_unstale":"2024-09-21T23:19:01.449028+0000","last_undegraded":"2024-09-21T23:19:01.449028+0000","last_fullsized":"2024-09-21T23:19:01.449028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'710","last_scrub_stamp":"2024-09-21T23:19:01.448837+0000","last_deep_scrub":"200'710","last_deep_scrub_stamp":"2024-09-21T23:19:01.448837+0000","last_clean_scrub_stamp":"2024-09-21T23:19:01.448837+0000","objects_scrubbed":14,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:24.150065+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0059155759999999996,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":12157,"num_read_kb":26818,"num_write":689,"num_write_kb":1770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":128054,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"200'419","reported_seq":5460,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:59.308799+0000","last_change":"2024-09-21T23:18:59.308851+0000","last_active":"2024-09-21T23:18:59.308799+0000","last_peered":"2024-09-21T23:18:59.308799+0000","last_clean":"2024-09-21T23:18:59.308799+0000","last_became_active":"2024-09-21T23:16:17.764014+0000","last_became_peered":"2024-09-21T23:16:17.764014+0000","last_unstale":"2024-09-21T23:18:59.308799+0000","last_undegraded":"2024-09-21T23:18:59.308799+0000","last_fullsized":"2024-09-21T23:18:59.308799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'419","last_scrub_stamp":"2024-09-21T23:18:59.308663+0000","last_deep_scrub":"200'419","last_deep_scrub_stamp":"2024-09-21T23:18:59.308663+0000","last_clean_scrub_stamp":"2024-09-21T23:18:59.308663+0000","objects_scrubbed":8,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:27.966640+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0016355880000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4922,"num_read_kb":66323,"num_write":323,"num_write_kb":33326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":138768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"200'679","reported_seq":18676,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:01.648682+0000","last_change":"2024-09-21T23:19:01.648889+0000","last_active":"2024-09-21T23:19:01.648682+0000","last_peered":"2024-09-21T23:19:01.648682+0000","last_clean":"2024-09-21T23:19:01.648682+0000","last_became_active":"2024-09-21T23:18:47.797088+0000","last_became_peered":"2024-09-21T23:18:47.797088+0000","last_unstale":"2024-09-21T23:19:01.648682+0000","last_undegraded":"2024-09-21T23:19:01.648682+0000","last_fullsized":"2024-09-21T23:19:01.648682+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'679","last_scrub_stamp":"2024-09-21T23:19:01.648464+0000","last_deep_scrub":"200'679","last_deep_scrub_stamp":"2024-09-21T23:19:01.648464+0000","last_clean_scrub_stamp":"2024-09-21T23:19:01.648464+0000","objects_scrubbed":7,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:42.577143+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001860163,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":16857,"num_read_kb":62840,"num_write":778,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":1243256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"200'428","reported_seq":13695,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:58.254308+0000","last_change":"2024-09-21T23:18:58.254538+0000","last_active":"2024-09-21T23:18:58.254308+0000","last_peered":"2024-09-21T23:18:58.254308+0000","last_clean":"2024-09-21T23:18:58.254308+0000","last_became_active":"2024-09-21T23:16:06.213656+0000","last_became_peered":"2024-09-21T23:16:06.213656+0000","last_unstale":"2024-09-21T23:18:58.254308+0000","last_undegraded":"2024-09-21T23:18:58.254308+0000","last_fullsized":"2024-09-21T23:18:58.254308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'428","last_scrub_stamp":"2024-09-21T23:18:58.254243+0000","last_deep_scrub":"200'428","last_deep_scrub_stamp":"2024-09-21T23:18:58.254243+0000","last_clean_scrub_stamp":"2024-09-21T23:18:58.254243+0000","objects_scrubbed":12,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:16:42.172910+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.003607828,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":13371,"num_read_kb":35460,"num_write":329,"num_write_kb":3459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":40512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"200'597","reported_seq":6270,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:06.588610+0000","last_change":"2024-09-21T23:19:06.588738+0000","last_active":"2024-09-21T23:19:06.588610+0000","last_peered":"2024-09-21T23:19:06.588610+0000","last_clean":"2024-09-21T23:19:06.588610+0000","last_became_active":"2024-09-21T23:18:39.798603+0000","last_became_peered":"2024-09-21T23:18:39.798603+0000","last_unstale":"2024-09-21T23:19:06.588610+0000","last_undegraded":"2024-09-21T23:19:06.588610+0000","last_fullsized":"2024-09-21T23:19:06.588610+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'597","last_scrub_stamp":"2024-09-21T23:19:06.588401+0000","last_deep_scrub":"200'597","last_deep_scrub_stamp":"2024-09-21T23:19:06.588401+0000","last_clean_scrub_stamp":"2024-09-21T23:19:06.588401+0000","objects_scrubbed":16,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:29:09.461754+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0061281979999999996,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5561,"num_read_kb":23483,"num_write":587,"num_write_kb":3551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":1570116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"200'353","reported_seq":4304,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:03.708546+0000","last_change":"2024-09-21T23:19:03.708642+0000","last_active":"2024-09-21T23:19:03.708546+0000","last_peered":"2024-09-21T23:19:03.708546+0000","last_clean":"2024-09-21T23:19:03.708546+0000","last_became_active":"2024-09-21T23:18:38.791983+0000","last_became_peered":"2024-09-21T23:18:38.791983+0000","last_unstale":"2024-09-21T23:19:03.708546+0000","last_undegraded":"2024-09-21T23:19:03.708546+0000","last_fullsized":"2024-09-21T23:19:03.708546+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'353","last_scrub_stamp":"2024-09-21T23:19:03.708367+0000","last_deep_scrub":"200'353","last_deep_scrub_stamp":"2024-09-21T23:19:03.708367+0000","last_clean_scrub_stamp":"2024-09-21T23:19:03.708367+0000","objects_scrubbed":4,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:26.972113+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0038304509999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3872,"num_read_kb":30392,"num_write":390,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":2192384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"200'942","reported_seq":8311,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:06.702312+0000","last_change":"2024-09-21T23:19:06.702392+0000","last_active":"2024-09-21T23:19:06.702312+0000","last_peered":"2024-09-21T23:19:06.702312+0000","last_clean":"2024-09-21T23:19:06.702312+0000","last_became_active":"2024-09-21T23:18:39.801495+0000","last_became_peered":"2024-09-21T23:18:39.801495+0000","last_unstale":"2024-09-21T23:19:06.702312+0000","last_undegraded":"2024-09-21T23:19:06.702312+0000","last_fullsized":"2024-09-21T23:19:06.702312+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'942","last_scrub_stamp":"2024-09-21T23:19:06.702090+0000","last_deep_scrub":"200'942","last_deep_scrub_stamp":"2024-09-21T23:19:06.702090+0000","last_clean_scrub_stamp":"2024-09-21T23:19:06.702090+0000","objects_scrubbed":4,"log_size":942,"log_dups_size":0,"ondisk_log_size":942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:48:17.876088+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0074401700000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11394,"num_read_kb":37180,"num_write":1083,"num_write_kb":13612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:01.718748+0000","last_change":"2024-09-21T23:19:01.718807+0000","last_active":"2024-09-21T23:19:01.718748+0000","last_peered":"2024-09-21T23:19:01.718748+0000","last_clean":"2024-09-21T23:19:01.718748+0000","last_became_active":"2024-09-21T23:18:33.986262+0000","last_became_peered":"2024-09-21T23:18:33.986262+0000","last_unstale":"2024-09-21T23:19:01.718748+0000","last_undegraded":"2024-09-21T23:19:01.718748+0000","last_fullsized":"2024-09-21T23:19:01.718748+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:19:01.718559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:19:01.718559+0000","last_clean_scrub_stamp":"2024-09-21T23:19:01.718559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:22:59.554934+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"200'774","reported_seq":20304,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:01.345950+0000","last_change":"2024-09-21T23:19:01.346130+0000","last_active":"2024-09-21T23:19:01.345950+0000","last_peered":"2024-09-21T23:19:01.345950+0000","last_clean":"2024-09-21T23:19:01.345950+0000","last_became_active":"2024-09-21T23:16:18.773528+0000","last_became_peered":"2024-09-21T23:16:18.773528+0000","last_unstale":"2024-09-21T23:19:01.345950+0000","last_undegraded":"2024-09-21T23:19:01.345950+0000","last_fullsized":"2024-09-21T23:19:01.345950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'774","last_scrub_stamp":"2024-09-21T23:19:01.345716+0000","last_deep_scrub":"200'774","last_deep_scrub_stamp":"2024-09-21T23:19:01.345716+0000","last_clean_scrub_stamp":"2024-09-21T23:19:01.345716+0000","objects_scrubbed":7,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:13:35.644224+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0027523489999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":20844,"num_read_kb":33891,"num_write":1018,"num_write_kb":4694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":2780752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":283,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:01.273985+0000","last_change":"2024-09-21T23:19:01.274044+0000","last_active":"2024-09-21T23:19:01.273985+0000","last_peered":"2024-09-21T23:19:01.273985+0000","last_clean":"2024-09-21T23:19:01.273985+0000","last_became_active":"2024-09-21T23:18:33.986227+0000","last_became_peered":"2024-09-21T23:18:33.986227+0000","last_unstale":"2024-09-21T23:19:01.273985+0000","last_undegraded":"2024-09-21T23:19:01.273985+0000","last_fullsized":"2024-09-21T23:19:01.273985+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:19:01.273778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:19:01.273778+0000","last_clean_scrub_stamp":"2024-09-21T23:19:01.273778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:41:54.037990+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00098526899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"200'1669","reported_seq":8621,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:02.672770+0000","last_change":"2024-09-21T23:19:02.672958+0000","last_active":"2024-09-21T23:19:02.672770+0000","last_peered":"2024-09-21T23:19:02.672770+0000","last_clean":"2024-09-21T23:19:02.672770+0000","last_became_active":"2024-09-21T23:18:48.795281+0000","last_became_peered":"2024-09-21T23:18:48.795281+0000","last_unstale":"2024-09-21T23:19:02.672770+0000","last_undegraded":"2024-09-21T23:19:02.672770+0000","last_fullsized":"2024-09-21T23:19:02.672770+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'1669","last_scrub_stamp":"2024-09-21T23:19:02.672531+0000","last_deep_scrub":"200'1669","last_deep_scrub_stamp":"2024-09-21T23:19:02.672531+0000","last_clean_scrub_stamp":"2024-09-21T23:19:02.672531+0000","objects_scrubbed":13,"log_size":1669,"log_dups_size":0,"ondisk_log_size":1669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:08.576611+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.001244768,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13386,"num_read_kb":49437,"num_write":2805,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4258004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:00.707358+0000","last_change":"2024-09-21T23:19:00.707411+0000","last_active":"2024-09-21T23:19:00.707358+0000","last_peered":"2024-09-21T23:19:00.707358+0000","last_clean":"2024-09-21T23:19:00.707358+0000","last_became_active":"2024-09-21T23:18:33.981926+0000","last_became_peered":"2024-09-21T23:18:33.981926+0000","last_unstale":"2024-09-21T23:19:00.707358+0000","last_undegraded":"2024-09-21T23:19:00.707358+0000","last_fullsized":"2024-09-21T23:19:00.707358+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:19:00.707170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:19:00.707170+0000","last_clean_scrub_stamp":"2024-09-21T23:19:00.707170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:00:53.477262+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"200'477","reported_seq":3331,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:03.297632+0000","last_change":"2024-09-21T23:19:03.297728+0000","last_active":"2024-09-21T23:19:03.297632+0000","last_peered":"2024-09-21T23:19:03.297632+0000","last_clean":"2024-09-21T23:19:03.297632+0000","last_became_active":"2024-09-21T23:18:47.794405+0000","last_became_peered":"2024-09-21T23:18:47.794405+0000","last_unstale":"2024-09-21T23:19:03.297632+0000","last_undegraded":"2024-09-21T23:19:03.297632+0000","last_fullsized":"2024-09-21T23:19:03.297632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'477","last_scrub_stamp":"2024-09-21T23:19:03.297442+0000","last_deep_scrub":"200'477","last_deep_scrub_stamp":"2024-09-21T23:19:03.297442+0000","last_clean_scrub_stamp":"2024-09-21T23:19:03.297442+0000","objects_scrubbed":14,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:37:26.755631+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.00073231999999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3043,"num_read_kb":11019,"num_write":472,"num_write_kb":1945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":181283,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":278,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:58.253752+0000","last_change":"2024-09-21T23:18:58.253786+0000","last_active":"2024-09-21T23:18:58.253752+0000","last_peered":"2024-09-21T23:18:58.253752+0000","last_clean":"2024-09-21T23:18:58.253752+0000","last_became_active":"2024-09-21T23:16:15.267304+0000","last_became_peered":"2024-09-21T23:16:15.267304+0000","last_unstale":"2024-09-21T23:18:58.253752+0000","last_undegraded":"2024-09-21T23:18:58.253752+0000","last_fullsized":"2024-09-21T23:18:58.253752+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:18:58.253652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:18:58.253652+0000","last_clean_scrub_stamp":"2024-09-21T23:18:58.253652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:35.865679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096156099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"200'572","reported_seq":11198,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:07.585396+0000","last_change":"2024-09-21T23:19:07.585396+0000","last_active":"2024-09-21T23:19:07.585396+0000","last_peered":"2024-09-21T23:19:07.585396+0000","last_clean":"2024-09-21T23:19:07.585396+0000","last_became_active":"2024-09-21T23:18:42.787666+0000","last_became_peered":"2024-09-21T23:18:42.787666+0000","last_unstale":"2024-09-21T23:19:07.585396+0000","last_undegraded":"2024-09-21T23:19:07.585396+0000","last_fullsized":"2024-09-21T23:19:07.585396+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'572","last_scrub_stamp":"2024-09-21T23:19:07.585117+0000","last_deep_scrub":"200'572","last_deep_scrub_stamp":"2024-09-21T23:19:07.585117+0000","last_clean_scrub_stamp":"2024-09-21T23:19:07.585117+0000","objects_scrubbed":13,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:18.993042+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0069917950000000003,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11469,"num_read_kb":76628,"num_write":643,"num_write_kb":6572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":407200,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"200'807","reported_seq":7145,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:00.465383+0000","last_change":"2024-09-21T23:19:00.465383+0000","last_active":"2024-09-21T23:19:00.465383+0000","last_peered":"2024-09-21T23:19:00.465383+0000","last_clean":"2024-09-21T23:19:00.465383+0000","last_became_active":"2024-09-21T23:16:06.214422+0000","last_became_peered":"2024-09-21T23:16:06.214422+0000","last_unstale":"2024-09-21T23:19:00.465383+0000","last_undegraded":"2024-09-21T23:19:00.465383+0000","last_fullsized":"2024-09-21T23:19:00.465383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'807","last_scrub_stamp":"2024-09-21T23:19:00.465095+0000","last_deep_scrub":"200'807","last_deep_scrub_stamp":"2024-09-21T23:19:00.465095+0000","last_clean_scrub_stamp":"2024-09-21T23:19:00.465095+0000","objects_scrubbed":15,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:34:01.867601+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0046990039999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8412,"num_read_kb":29133,"num_write":771,"num_write_kb":5629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":111134,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"200'3","reported_seq":415,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:04.577473+0000","last_change":"2024-09-21T23:19:04.577473+0000","last_active":"2024-09-21T23:19:04.577473+0000","last_peered":"2024-09-21T23:19:04.577473+0000","last_clean":"2024-09-21T23:19:04.577473+0000","last_became_active":"2024-09-21T23:16:15.264129+0000","last_became_peered":"2024-09-21T23:16:15.264129+0000","last_unstale":"2024-09-21T23:19:04.577473+0000","last_undegraded":"2024-09-21T23:19:04.577473+0000","last_fullsized":"2024-09-21T23:19:04.577473+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'3","last_scrub_stamp":"2024-09-21T23:19:04.577209+0000","last_deep_scrub":"200'3","last_deep_scrub_stamp":"2024-09-21T23:19:04.577209+0000","last_clean_scrub_stamp":"2024-09-21T23:19:04.577209+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:22:05.379772+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0022026329999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"200'380","reported_seq":6700,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:04.246176+0000","last_change":"2024-09-21T23:19:04.246252+0000","last_active":"2024-09-21T23:19:04.246176+0000","last_peered":"2024-09-21T23:19:04.246176+0000","last_clean":"2024-09-21T23:19:04.246176+0000","last_became_active":"2024-09-21T23:18:47.798894+0000","last_became_peered":"2024-09-21T23:18:47.798894+0000","last_unstale":"2024-09-21T23:19:04.246176+0000","last_undegraded":"2024-09-21T23:19:04.246176+0000","last_fullsized":"2024-09-21T23:19:04.246176+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'380","last_scrub_stamp":"2024-09-21T23:19:04.245980+0000","last_deep_scrub":"200'380","last_deep_scrub_stamp":"2024-09-21T23:19:04.245980+0000","last_clean_scrub_stamp":"2024-09-21T23:19:04.245980+0000","objects_scrubbed":9,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:26:28.727225+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0015079399999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5910,"num_read_kb":24014,"num_write":310,"num_write_kb":1768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":148216,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:58.707494+0000","last_change":"2024-09-21T23:18:58.707534+0000","last_active":"2024-09-21T23:18:58.707494+0000","last_peered":"2024-09-21T23:18:58.707494+0000","last_clean":"2024-09-21T23:18:58.707494+0000","last_became_active":"2024-09-21T23:18:42.788015+0000","last_became_peered":"2024-09-21T23:18:42.788015+0000","last_unstale":"2024-09-21T23:18:58.707494+0000","last_undegraded":"2024-09-21T23:18:58.707494+0000","last_fullsized":"2024-09-21T23:18:58.707494+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:18:58.707348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:18:58.707348+0000","last_clean_scrub_stamp":"2024-09-21T23:18:58.707348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:04:35.788504+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00036505199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"200'434","reported_seq":4810,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:59.253123+0000","last_change":"2024-09-21T23:18:59.253169+0000","last_active":"2024-09-21T23:18:59.253123+0000","last_peered":"2024-09-21T23:18:59.253123+0000","last_clean":"2024-09-21T23:18:59.253123+0000","last_became_active":"2024-09-21T23:16:06.218486+0000","last_became_peered":"2024-09-21T23:16:06.218486+0000","last_unstale":"2024-09-21T23:18:59.253123+0000","last_undegraded":"2024-09-21T23:18:59.253123+0000","last_fullsized":"2024-09-21T23:18:59.253123+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'434","last_scrub_stamp":"2024-09-21T23:18:59.253006+0000","last_deep_scrub":"200'434","last_deep_scrub_stamp":"2024-09-21T23:18:59.253006+0000","last_clean_scrub_stamp":"2024-09-21T23:18:59.253006+0000","objects_scrubbed":9,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:29:53.405958+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012632159999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3934,"num_read_kb":15474,"num_write":377,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1303588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:18:59.656667+0000","last_change":"2024-09-21T23:18:59.656722+0000","last_active":"2024-09-21T23:18:59.656667+0000","last_peered":"2024-09-21T23:18:59.656667+0000","last_clean":"2024-09-21T23:18:59.656667+0000","last_became_active":"2024-09-21T23:18:33.977449+0000","last_became_peered":"2024-09-21T23:18:33.977449+0000","last_unstale":"2024-09-21T23:18:59.656667+0000","last_undegraded":"2024-09-21T23:18:59.656667+0000","last_fullsized":"2024-09-21T23:18:59.656667+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T23:18:59.656476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T23:18:59.656476+0000","last_clean_scrub_stamp":"2024-09-21T23:18:59.656476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:16:31.511812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"200'669","reported_seq":17058,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:00.325843+0000","last_change":"2024-09-21T23:19:00.325915+0000","last_active":"2024-09-21T23:19:00.325843+0000","last_peered":"2024-09-21T23:19:00.325843+0000","last_clean":"2024-09-21T23:19:00.325843+0000","last_became_active":"2024-09-21T23:16:17.764401+0000","last_became_peered":"2024-09-21T23:16:17.764401+0000","last_unstale":"2024-09-21T23:19:00.325843+0000","last_undegraded":"2024-09-21T23:19:00.325843+0000","last_fullsized":"2024-09-21T23:19:00.325843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'669","last_scrub_stamp":"2024-09-21T23:19:00.325729+0000","last_deep_scrub":"200'669","last_deep_scrub_stamp":"2024-09-21T23:19:00.325729+0000","last_clean_scrub_stamp":"2024-09-21T23:19:00.325729+0000","objects_scrubbed":18,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:22:19.249811+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0020263540000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18824,"num_read_kb":53498,"num_write":527,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":1178797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"200'2","reported_seq":411,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:08.564099+0000","last_change":"2024-09-21T23:19:08.564167+0000","last_active":"2024-09-21T23:19:08.564099+0000","last_peered":"2024-09-21T23:19:08.564099+0000","last_clean":"2024-09-21T23:19:08.564099+0000","last_became_active":"2024-09-21T23:18:42.788526+0000","last_became_peered":"2024-09-21T23:18:42.788526+0000","last_unstale":"2024-09-21T23:19:08.564099+0000","last_undegraded":"2024-09-21T23:19:08.564099+0000","last_fullsized":"2024-09-21T23:19:08.564099+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'2","last_scrub_stamp":"2024-09-21T23:19:08.563896+0000","last_deep_scrub":"200'2","last_deep_scrub_stamp":"2024-09-21T23:19:08.563896+0000","last_clean_scrub_stamp":"2024-09-21T23:19:08.563896+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:05:48.988837+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00077169399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"200'119","reported_seq":494,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:02.701771+0000","last_change":"2024-09-21T23:19:02.701912+0000","last_active":"2024-09-21T23:19:02.701771+0000","last_peered":"2024-09-21T23:19:02.701771+0000","last_clean":"2024-09-21T23:19:02.701771+0000","last_became_active":"2024-09-21T23:18:36.777596+0000","last_became_peered":"2024-09-21T23:18:36.777596+0000","last_unstale":"2024-09-21T23:19:02.701771+0000","last_undegraded":"2024-09-21T23:19:02.701771+0000","last_fullsized":"2024-09-21T23:19:02.701771+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'119","last_scrub_stamp":"2024-09-21T23:19:02.701465+0000","last_deep_scrub":"200'119","last_deep_scrub_stamp":"2024-09-21T23:19:02.701465+0000","last_clean_scrub_stamp":"2024-09-21T23:19:02.701465+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:58:40.293485+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"200'917","reported_seq":12950,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:00.664712+0000","last_change":"2024-09-21T23:19:00.664806+0000","last_active":"2024-09-21T23:19:00.664712+0000","last_peered":"2024-09-21T23:19:00.664712+0000","last_clean":"2024-09-21T23:19:00.664712+0000","last_became_active":"2024-09-21T23:18:47.793978+0000","last_became_peered":"2024-09-21T23:18:47.793978+0000","last_unstale":"2024-09-21T23:19:00.664712+0000","last_undegraded":"2024-09-21T23:19:00.664712+0000","last_fullsized":"2024-09-21T23:19:00.664712+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'917","last_scrub_stamp":"2024-09-21T23:19:00.664513+0000","last_deep_scrub":"200'917","last_deep_scrub_stamp":"2024-09-21T23:19:00.664513+0000","last_clean_scrub_stamp":"2024-09-21T23:19:00.664513+0000","objects_scrubbed":13,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:21:37.409036+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0011535569999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13624,"num_read_kb":34943,"num_write":770,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":461888,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"200'606","reported_seq":5324,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:02.436315+0000","last_change":"2024-09-21T23:19:02.436407+0000","last_active":"2024-09-21T23:19:02.436315+0000","last_peered":"2024-09-21T23:19:02.436315+0000","last_clean":"2024-09-21T23:19:02.436315+0000","last_became_active":"2024-09-21T23:18:46.787797+0000","last_became_peered":"2024-09-21T23:18:46.787797+0000","last_unstale":"2024-09-21T23:19:02.436315+0000","last_undegraded":"2024-09-21T23:19:02.436315+0000","last_fullsized":"2024-09-21T23:19:02.436315+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'606","last_scrub_stamp":"2024-09-21T23:19:02.436107+0000","last_deep_scrub":"200'606","last_deep_scrub_stamp":"2024-09-21T23:19:02.436107+0000","last_clean_scrub_stamp":"2024-09-21T23:19:02.436107+0000","objects_scrubbed":14,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:17.525108+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0085867570000000004,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6138,"num_read_kb":14831,"num_write":469,"num_write_kb":2893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":111616,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"200'365","reported_seq":4280,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:05.720472+0000","last_change":"2024-09-21T23:19:05.720636+0000","last_active":"2024-09-21T23:19:05.720472+0000","last_peered":"2024-09-21T23:19:05.720472+0000","last_clean":"2024-09-21T23:19:05.720472+0000","last_became_active":"2024-09-21T23:18:38.795761+0000","last_became_peered":"2024-09-21T23:18:38.795761+0000","last_unstale":"2024-09-21T23:19:05.720472+0000","last_undegraded":"2024-09-21T23:19:05.720472+0000","last_fullsized":"2024-09-21T23:19:05.720472+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'365","last_scrub_stamp":"2024-09-21T23:19:05.720185+0000","last_deep_scrub":"200'365","last_deep_scrub_stamp":"2024-09-21T23:19:05.720185+0000","last_clean_scrub_stamp":"2024-09-21T23:19:05.720185+0000","objects_scrubbed":11,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:40:08.824188+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0046901069999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":3321,"num_read_kb":18181,"num_write":294,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":76304,"num_keys_recovered":15,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"200'797","reported_seq":8684,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:02.346562+0000","last_change":"2024-09-21T23:19:02.346634+0000","last_active":"2024-09-21T23:19:02.346562+0000","last_peered":"2024-09-21T23:19:02.346562+0000","last_clean":"2024-09-21T23:19:02.346562+0000","last_became_active":"2024-09-21T23:17:37.171896+0000","last_became_peered":"2024-09-21T23:17:37.171896+0000","last_unstale":"2024-09-21T23:19:02.346562+0000","last_undegraded":"2024-09-21T23:19:02.346562+0000","last_fullsized":"2024-09-21T23:19:02.346562+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'797","last_scrub_stamp":"2024-09-21T23:19:02.346374+0000","last_deep_scrub":"200'797","last_deep_scrub_stamp":"2024-09-21T23:19:02.346374+0000","last_clean_scrub_stamp":"2024-09-21T23:19:02.346374+0000","objects_scrubbed":6,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:45.308963+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0029267189999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10899,"num_read_kb":22573,"num_write":696,"num_write_kb":2123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":1001083,"num_keys_recovered":81,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"200'696","reported_seq":6175,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:02.325002+0000","last_change":"2024-09-21T23:19:02.325087+0000","last_active":"2024-09-21T23:19:02.325002+0000","last_peered":"2024-09-21T23:19:02.325002+0000","last_clean":"2024-09-21T23:19:02.325002+0000","last_became_active":"2024-09-21T23:18:42.788002+0000","last_became_peered":"2024-09-21T23:18:42.788002+0000","last_unstale":"2024-09-21T23:19:02.325002+0000","last_undegraded":"2024-09-21T23:19:02.325002+0000","last_fullsized":"2024-09-21T23:19:02.325002+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'696","last_scrub_stamp":"2024-09-21T23:19:02.324818+0000","last_deep_scrub":"200'696","last_deep_scrub_stamp":"2024-09-21T23:19:02.324818+0000","last_clean_scrub_stamp":"2024-09-21T23:19:02.324818+0000","objects_scrubbed":10,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:42:56.620427+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0062598719999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7976,"num_read_kb":18238,"num_write":516,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":81408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"200'482","reported_seq":5567,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:08.662983+0000","last_change":"2024-09-21T23:19:08.663065+0000","last_active":"2024-09-21T23:19:08.662983+0000","last_peered":"2024-09-21T23:19:08.662983+0000","last_clean":"2024-09-21T23:19:08.662983+0000","last_became_active":"2024-09-21T23:18:46.786927+0000","last_became_peered":"2024-09-21T23:18:46.786927+0000","last_unstale":"2024-09-21T23:19:08.662983+0000","last_undegraded":"2024-09-21T23:19:08.662983+0000","last_fullsized":"2024-09-21T23:19:08.662983+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'482","last_scrub_stamp":"2024-09-21T23:19:08.662798+0000","last_deep_scrub":"200'482","last_deep_scrub_stamp":"2024-09-21T23:19:08.662798+0000","last_clean_scrub_stamp":"2024-09-21T23:19:08.662798+0000","objects_scrubbed":7,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:05:45.828654+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.003514428,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6210,"num_read_kb":29417,"num_write":537,"num_write_kb":1711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":195340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"200'479","reported_seq":4635,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:02.573464+0000","last_change":"2024-09-21T23:19:02.573543+0000","last_active":"2024-09-21T23:19:02.573464+0000","last_peered":"2024-09-21T23:19:02.573464+0000","last_clean":"2024-09-21T23:19:02.573464+0000","last_became_active":"2024-09-21T23:15:01.313078+0000","last_became_peered":"2024-09-21T23:15:01.313078+0000","last_unstale":"2024-09-21T23:19:02.573464+0000","last_undegraded":"2024-09-21T23:19:02.573464+0000","last_fullsized":"2024-09-21T23:19:02.573464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'479","last_scrub_stamp":"2024-09-21T23:19:02.573284+0000","last_deep_scrub":"200'479","last_deep_scrub_stamp":"2024-09-21T23:19:02.573284+0000","last_clean_scrub_stamp":"2024-09-21T23:19:02.573284+0000","objects_scrubbed":7,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:07.680151+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0073184640000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5566,"num_read_kb":14930,"num_write":412,"num_write_kb":2516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"200'493","reported_seq":6855,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:05.549855+0000","last_change":"2024-09-21T23:19:05.549953+0000","last_active":"2024-09-21T23:19:05.549855+0000","last_peered":"2024-09-21T23:19:05.549855+0000","last_clean":"2024-09-21T23:19:05.549855+0000","last_became_active":"2024-09-21T23:18:36.784584+0000","last_became_peered":"2024-09-21T23:18:36.784584+0000","last_unstale":"2024-09-21T23:19:05.549855+0000","last_undegraded":"2024-09-21T23:19:05.549855+0000","last_fullsized":"2024-09-21T23:19:05.549855+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'493","last_scrub_stamp":"2024-09-21T23:19:05.549670+0000","last_deep_scrub":"200'493","last_deep_scrub_stamp":"2024-09-21T23:19:05.549670+0000","last_clean_scrub_stamp":"2024-09-21T23:19:05.549670+0000","objects_scrubbed":14,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:19:38.949302+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0080862480000000007,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7931,"num_read_kb":34801,"num_write":430,"num_write_kb":4977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":140816,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"200'784","reported_seq":6216,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:03.620572+0000","last_change":"2024-09-21T23:19:03.620653+0000","last_active":"2024-09-21T23:19:03.620572+0000","last_peered":"2024-09-21T23:19:03.620572+0000","last_clean":"2024-09-21T23:19:03.620572+0000","last_became_active":"2024-09-21T23:16:06.209024+0000","last_became_peered":"2024-09-21T23:16:06.209024+0000","last_unstale":"2024-09-21T23:19:03.620572+0000","last_undegraded":"2024-09-21T23:19:03.620572+0000","last_fullsized":"2024-09-21T23:19:03.620572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'784","last_scrub_stamp":"2024-09-21T23:19:03.620368+0000","last_deep_scrub":"200'784","last_deep_scrub_stamp":"2024-09-21T23:19:03.620368+0000","last_clean_scrub_stamp":"2024-09-21T23:19:03.620368+0000","objects_scrubbed":8,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:47:12.612332+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0028297349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7092,"num_read_kb":28624,"num_write":923,"num_write_kb":6046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":121856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"200'547","reported_seq":5589,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:07.667893+0000","last_change":"2024-09-21T23:19:07.667974+0000","last_active":"2024-09-21T23:19:07.667893+0000","last_peered":"2024-09-21T23:19:07.667893+0000","last_clean":"2024-09-21T23:19:07.667893+0000","last_became_active":"2024-09-21T23:18:40.802654+0000","last_became_peered":"2024-09-21T23:18:40.802654+0000","last_unstale":"2024-09-21T23:19:07.667893+0000","last_undegraded":"2024-09-21T23:19:07.667893+0000","last_fullsized":"2024-09-21T23:19:07.667893+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'547","last_scrub_stamp":"2024-09-21T23:19:07.667699+0000","last_deep_scrub":"200'547","last_deep_scrub_stamp":"2024-09-21T23:19:07.667699+0000","last_clean_scrub_stamp":"2024-09-21T23:19:07.667699+0000","objects_scrubbed":8,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:45:13.244251+0000","scrub_duration":15,"objects_trimmed":2,"snaptrim_duration":0.0029604689999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5656,"num_read_kb":28845,"num_write":648,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":134181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"200'815","reported_seq":22316,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:04.340325+0000","last_change":"2024-09-21T23:19:04.340400+0000","last_active":"2024-09-21T23:19:04.340325+0000","last_peered":"2024-09-21T23:19:04.340325+0000","last_clean":"2024-09-21T23:19:04.340325+0000","last_became_active":"2024-09-21T23:18:40.816843+0000","last_became_peered":"2024-09-21T23:18:40.816843+0000","last_unstale":"2024-09-21T23:19:04.340325+0000","last_undegraded":"2024-09-21T23:19:04.340325+0000","last_fullsized":"2024-09-21T23:19:04.340325+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'815","last_scrub_stamp":"2024-09-21T23:19:04.340141+0000","last_deep_scrub":"200'815","last_deep_scrub_stamp":"2024-09-21T23:19:04.340141+0000","last_clean_scrub_stamp":"2024-09-21T23:19:04.340141+0000","objects_scrubbed":8,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:33:59.760060+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.003015551,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":24770,"num_read_kb":46884,"num_write":769,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":300152,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"200'496","reported_seq":9851,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-21T23:19:04.705056+0000","last_change":"2024-09-21T23:19:04.705161+0000","last_active":"2024-09-21T23:19:04.705056+0000","last_peered":"2024-09-21T23:19:04.705056+0000","last_clean":"2024-09-21T23:19:04.705056+0000","last_became_active":"2024-09-21T23:18:38.791632+0000","last_became_peered":"2024-09-21T23:18:38.791632+0000","last_unstale":"2024-09-21T23:19:04.705056+0000","last_undegraded":"2024-09-21T23:19:04.705056+0000","last_fullsized":"2024-09-21T23:19:04.705056+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"200'496","last_scrub_stamp":"2024-09-21T23:19:04.704837+0000","last_deep_scrub":"200'496","last_deep_scrub_stamp":"2024-09-21T23:19:04.704837+0000","last_clean_scrub_stamp":"2024-09-21T23:19:04.704837+0000","objects_scrubbed":12,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:58.443930+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0025997469999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9412,"num_read_kb":22497,"num_write":527,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":160272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":330,"num_read":313855,"num_read_kb":1079349,"num_write":21231,"num_write_kb":208209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4344,"num_bytes_recovered":23171511,"num_keys_recovered":317,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},"log_size":20593,"ondisk_log_size":20593,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607611,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51584,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320256,"statfs":{"total":96636764160,"available":96583942144,"internally_reserved":0,"allocated":2408448,"data_stored":1008181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607611,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55744,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316096,"statfs":{"total":96636764160,"available":96579682304,"internally_reserved":0,"allocated":2408448,"data_stored":1008181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607611,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42432,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329408,"statfs":{"total":96636764160,"available":96593313792,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607611,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51580,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320260,"statfs":{"total":96636764160,"available":96583946240,"internally_reserved":0,"allocated":2404352,"data_stored":1008162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607612,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77956,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293884,"statfs":{"total":96636764160,"available":96556937216,"internally_reserved":0,"allocated":2412544,"data_stored":1008200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":61,"seq":261993005097,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61632,"kb_used_data":2928,"kb_used_omap":5,"kb_used_meta":58682,"kb_avail":94310208,"statfs":{"total":96636764160,"available":96573652992,"internally_reserved":0,"allocated":2998272,"data_stored":1598530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5851,"internal_metadata":60090661},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3064,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T23:19:20.841 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T23:19:21.172 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T23:19:21.172 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T23:19:21.173 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T23:19:21.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:21.240 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T23:19:21.240 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T23:19:21.240 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:21.312 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T23:19:21.312 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T23:19:21.312 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:21.394 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T23:19:21.395 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T23:19:21.395 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:21.485 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T23:19:21.486 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T23:19:21.486 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:25.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-21T23:19:27.588 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T23:19:27.588 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T23:19:27.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:27.659 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T23:19:27.659 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T23:19:27.659 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T23:19:27.659 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:27.689 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T23:19:27.689 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T23:19:27.690 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T23:19:27.690 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:27.715 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T23:19:27.715 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T23:19:27.715 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:27.776 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T23:19:27.777 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T23:19:27.777 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T23:19:27.793 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T23:19:27.793 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T23:19:27.793 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-21T23:19:27.837 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:19:27.837 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T23:19:28.273 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:19:28.273 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T23:19:28.447 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:19:28.448 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T23:19:28.589 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:19:28.590 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T23:19:28.879 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:19:28.880 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T23:19:29.116 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:19:29.117 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T23:19:29.311 INFO:tasks.ceph:Archiving mon data... 2024-09-21T23:19:29.312 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/data/mon.a.tgz 2024-09-21T23:19:29.312 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-21T23:19:29.318 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.WzH0quYhpv 2024-09-21T23:19:29.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WzH0quYhpv 2024-09-21T23:19:29.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.WzH0quYhpv 2024-09-21T23:19:29.527 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.WzH0quYhpv is 378KB 2024-09-21T23:19:29.584 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.WzH0quYhpv 2024-09-21T23:19:29.590 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/data/mon.c.tgz 2024-09-21T23:19:29.590 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-21T23:19:29.635 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.wIPc51x2s4 2024-09-21T23:19:29.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wIPc51x2s4 2024-09-21T23:19:29.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.wIPc51x2s4 2024-09-21T23:19:29.886 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.wIPc51x2s4 is 520KB 2024-09-21T23:19:29.952 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.wIPc51x2s4 2024-09-21T23:19:29.957 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/data/mon.b.tgz 2024-09-21T23:19:29.957 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-09-21T23:19:29.963 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.T9rqunEauc 2024-09-21T23:19:29.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.T9rqunEauc 2024-09-21T23:19:30.135 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.T9rqunEauc 2024-09-21T23:19:30.198 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.T9rqunEauc is 520KB 2024-09-21T23:19:30.260 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.T9rqunEauc 2024-09-21T23:19:30.265 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T23:19:30.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:19:30.267 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:19:30.307 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T23:19:30.327 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:19:30.366 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:19:30.371 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T23:19:30.371 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T23:19:30.373 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi053/crash 2024-09-21T23:19:30.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:19:30.388 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi144/crash 2024-09-21T23:19:30.389 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:19:30.423 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi170/crash 2024-09-21T23:19:30.424 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T23:19:30.437 INFO:tasks.ceph:Compressing logs... 2024-09-21T23:19:30.437 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:19:30.440 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:19:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2024-09-21T23:19:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42880.log 2024-09-21T23:19:30.457 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz 2024-09-21T23:19:30.457 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40410.log 2024-09-21T23:19:30.457 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42119.log 2024-09-21T23:19:30.457 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42880.log.gz 2024-09-21T23:19:30.458 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40675.log 2024-09-21T23:19:30.459 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40410.log.gz 2024-09-21T23:19:30.459 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42119.log.gz 2024-09-21T23:19:30.459 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42869.log 2024-09-21T23:19:30.459 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40675.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.29691.log 2024-09-21T23:19:30.459 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40675.log.gz 2024-09-21T23:19:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42869.log.gz 2024-09-21T23:19:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33401.log 2024-09-21T23:19:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.29691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43876.log 2024-09-21T23:19:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33401.log.gzgzip 2024-09-21T23:19:30.460 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38680.log 2024-09-21T23:19:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42221.log 2024-09-21T23:19:30.461 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43876.log.gz 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38680.log.gz 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41143.log 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30722.log 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42221.log.gz 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41143.log.gz 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32151.log 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.30722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38088.log 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30722.log.gz 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.32151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32151.log.gz 2024-09-21T23:19:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33640.log 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28368.log 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38088.log.gz 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33640.log.gz 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28368.log.gz 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35913.log 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28684.log 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35913.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.29691.log.gz 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29498.log 2024-09-21T23:19:30.465 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35913.log.gz 2024-09-21T23:19:30.468 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.28684.log: /var/log/ceph/ceph-client.admin.37230.log 2024-09-21T23:19:30.468 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28684.log.gz 2024-09-21T23:19:30.468 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29498.log.gz 2024-09-21T23:19:30.468 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29010.log 2024-09-21T23:19:30.468 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2024-09-21T23:19:30.468 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37230.log.gz 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38474.log 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29010.log.gz 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37117.log 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43564.log 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38474.log.gz 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 0.0% 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37117.log.gz 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.43564.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29597.log 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43564.log.gz 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-09-21T23:19:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2024-09-21T23:19:30.470 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38171.log 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29597.log.gz 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38833.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz -- 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.33461.log 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41448.log 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38171.log.gz 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43356.log 2024-09-21T23:19:30.476 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33461.log.gz 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41448.log.gz 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36627.log 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40140.log 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43356.log.gz 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35437.log 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36627.log.gz 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40140.log.gz 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43590.log 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35437.log.gz 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38034.log 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34341.log 2024-09-21T23:19:30.477 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43590.log.gz 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43772.log 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38034.log.gz 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42322.log 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34341.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34341.log.gz 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41148.log 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43772.log.gz 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42322.log.gz 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28908.log 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41338.log 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41148.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41148.log.gz 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28908.log.gz 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35516.log 2024-09-21T23:19:30.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29170.log 2024-09-21T23:19:30.479 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41338.log.gz 2024-09-21T23:19:30.479 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38440.log 0.0% 2024-09-21T23:19:30.479 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35516.log.gz 2024-09-21T23:19:30.479 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-09-21T23:19:30.479 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29170.log.gz 2024-09-21T23:19:30.479 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38440.log.gz 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25211.log 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25307.log 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.25211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25259.log 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr: 27.6% -- replaced with /var/log/ceph/ceph-client.admin.25211.log.gz 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.25307.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25307.log.gz 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.25259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25259.log.gz 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T23:19:30.493 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25235.log 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.25235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25187.log 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25235.log.gz 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.25187.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25187.log.gz 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.25283.log 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.25283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25283.log.gz 2024-09-21T23:19:30.494 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.4.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T23:19:30.576 INFO:teuthology.orchestra.run.smithi144.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T23:19:30.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23167.log 2024-09-21T23:19:30.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21944.log 2024-09-21T23:19:30.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21663.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.23167.log.gz 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21944.log: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.21944.log.gz 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23564.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23424.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21663.log: /var/log/ceph/ceph-client.admin.23564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22075.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.21663.log.gz 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.23564.log.gz 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21874.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.23424.log.gz 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22966.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22075.log.gz 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23210.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.21874.log.gz 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21935.log 2024-09-21T23:19:30.597 INFO:teuthology.orchestra.run.smithi170.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.22966.log.gz 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24020.log 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr: 75.5% -- replaced with /var/log/ceph/ceph-client.admin.23210.log.gz 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21935.log: gzip -5 --verbose -- 19.4% /var/log/ceph/ceph-client.admin.24151.log 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.21935.log.gz 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22887.log 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.24020.log.gz 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22181.log 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24151.log.gz 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23914.log 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22887.log.gz 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21585.log 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22181.log.gz 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23237.log 2024-09-21T23:19:30.598 INFO:teuthology.orchestra.run.smithi170.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.23914.log.gz 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23176.log 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21585.log.gz 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22532.log 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.23237.log.gz 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22496.log 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24054.log 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22532.log.gz 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.admin.23176.log.gz 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22120.log 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22496.log.gz 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23801.log 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.24054.log.gz 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23451.log 2024-09-21T23:19:30.599 INFO:teuthology.orchestra.run.smithi170.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.22120.log.gz 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.23801.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.21971.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23801.log.gz 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24361.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23451.log.gz 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22224.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21971.log.gz 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23000.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24361.log.gz 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22224.log: gzip -5 24.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.22224.log.gz /var/log/ceph/ceph-client.admin.24221.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21620.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.23000.log.gz 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22014.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24221.log.gz 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23871.log 2024-09-21T23:19:30.600 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21620.log.gz 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22789.log 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr: 34.6% -- replaced with /var/log/ceph/ceph-client.admin.22014.log.gz 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.23871.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.23774.log 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.23871.log.gz 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22746.log 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.22789.log.gz 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24090.log 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23774.log.gz 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22294.log 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22746.log.gz 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24124.log 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.24090.log.gz 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22957.log 2024-09-21T23:19:30.601 INFO:teuthology.orchestra.run.smithi170.stderr: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.22294.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22645.log 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.24124.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22957.log: gzip -5 --verbose -- 19.4% /var/log/ceph/ceph-client.admin.24300.log -- replaced with /var/log/ceph/ceph-client.admin.22957.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23521.log 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.22645.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.24300.log: /var/log/ceph/ceph-client.admin.24081.log 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23521.log: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.24300.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23905.log 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23521.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23345.log 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24081.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22251.log 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23905.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23905.log.gz 2024-09-21T23:19:30.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23354.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23345.log.gz 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21594.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22251.log.gz 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22921.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.23354.log.gz 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22154.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.21594.log.gz 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22684.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22921.log.gz 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22718.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22154.log: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.22154.log.gz 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21700.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.admin.22684.log.gz 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23844.log 2024-09-21T23:19:30.603 INFO:teuthology.orchestra.run.smithi170.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.22718.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22896.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21700.log: /var/log/ceph/ceph-client.admin.23844.log: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.21700.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22400.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.23844.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21901.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22400.log: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.22896.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23740.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21901.log: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.22400.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24160.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21901.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21523.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.23740.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24255.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21523.log: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.24160.log.gz 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21654.log 2024-09-21T23:19:30.604 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21523.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21840.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24255.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.21654.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.23246.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21654.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22260.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.21840.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21865.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22260.log: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.23246.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24230.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.22260.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22471.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21865.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21831.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.24230.log.gz 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23835.log 2024-09-21T23:19:30.605 INFO:teuthology.orchestra.run.smithi170.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.22471.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23634.log 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21831.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23027.log 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23835.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5/var/log/ceph/ceph-client.admin.23634.log: --verbose -- /var/log/ceph/ceph-client.admin.23941.log 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr: 52.1% -- replaced with /var/log/ceph/ceph-client.admin.23634.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21734.log 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23027.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23530.log 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23941.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.21734.log: /var/log/ceph/ceph-client.admin.22991.log 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.21734.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22755.log 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.23530.log.gz 2024-09-21T23:19:30.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23061.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22991.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23975.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22755.log: /var/log/ceph/ceph-client.admin.23061.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.22755.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22330.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23061.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21795.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.23975.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23271.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21795.log: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.22330.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21795.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22355.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21910.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23271.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21770.log 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22355.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22355.log.gz 2024-09-21T23:19:30.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21725.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.21910.log.gz 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.21499.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.21770.log.gz 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21804.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21725.log: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.21725.log.gz 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.21499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24291.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.21499.log.gz 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22050.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.21804.log.gz 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22190.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24291.log: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24291.log.gz 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24115.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.22050.log.gz 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23320.log 2024-09-21T23:19:30.608 INFO:teuthology.orchestra.run.smithi170.stderr: 52.1% -- replaced with /var/log/ceph/ceph-client.admin.22190.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22636.log 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.24115.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22005.log 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.23320.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22285.log 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22636.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22817.log 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22005.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22005.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22826.log 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22285.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23201.log 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.22817.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22675.log 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22826.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.22826.log.gz 2024-09-21T23:19:30.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23311.log 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.23201.log.gz 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24264.log 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22675.log.gz 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23765.log 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23311.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23311.log.gz 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23661.log 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.24264.log.gz 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22541.log 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23765.log.gz 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22602.log 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23661.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23661.log.gz 2024-09-21T23:19:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22566.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.22541.log.gz 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22215.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22602.log.gz 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22611.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22566.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22566.log.gz 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23695.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22215.log.gz 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23600.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.22611.log.gz 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23984.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23695.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23695.log.gz 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24011.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.23600.log.gz 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23731.log 2024-09-21T23:19:30.611 INFO:teuthology.orchestra.run.smithi170.stderr: 55.5% -- replaced with /var/log/ceph/ceph-client.admin.23984.log.gz 2024-09-21T23:19:30.612 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23555.log 2024-09-21T23:19:30.612 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24011.log: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.24011.log.gz 2024-09-21T23:19:30.612 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23140.log 2024-09-21T23:19:30.613 INFO:teuthology.orchestra.run.smithi170.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.23731.log.gz 2024-09-21T23:19:30.613 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23280.log 2024-09-21T23:19:30.613 INFO:teuthology.orchestra.run.smithi170.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.23555.log.gz 2024-09-21T23:19:30.613 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22391.log 2024-09-21T23:19:30.613 INFO:teuthology.orchestra.run.smithi170.stderr: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.23140.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23485.log 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr: 56.2% -- replaced with /var/log/ceph/ceph-client.admin.23280.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23415.log 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22391.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23494.log 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23485.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23485.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23131.log 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23415.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23381.log 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23494.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22575.log 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23131.log: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.23131.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23704.log 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23381.log.gz 2024-09-21T23:19:30.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22505.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.22575.log.gz 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22364.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23704.log: /var/log/ceph/ceph-client.admin.22505.log: 54.9% -- replaced with /var/log/ceph/ceph-client.admin.23704.log.gz 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24325.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.22505.log.gz 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22860.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.22364.log.gz 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24334.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.24325.log.gz 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22709.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.22860.log.gz 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22425.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.24334.log.gz 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22084.log 2024-09-21T23:19:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22709.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22709.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22851.log 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22425.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21761.log 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.22084.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21629.log 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22851.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22851.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22145.log 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21761.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24194.log 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.22145.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.22462.log 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22145.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22041.log 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.24194.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.21629.log.gz 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22462.log: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22462.log.gzgzip 2024-09-21T23:19:30.616 INFO:teuthology.orchestra.run.smithi170.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.23810.log 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21691.log 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22041.log.gz 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22321.log 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.23810.log.gz 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23070.log 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21691.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.21691.log.gz 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23670.log 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.22321.log.gz 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23591.log 2024-09-21T23:19:30.617 INFO:teuthology.orchestra.run.smithi170.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.23070.log.gz 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24045.log 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.23670.log.gz 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23390.log 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23591.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.23591.log.gz 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23625.log 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.24045.log.gz 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22930.log 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.23390.log.gz 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24185.log 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23625.log: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.23625.log.gz 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23036.log 2024-09-21T23:19:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22930.log: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.22930.log.gz 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24185.log: gzip 20.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.24185.log.gz --verbose 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr: -- /var/log/ceph/ceph-client.admin.24370.log 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23097.log 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23036.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.23036.log.gz 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.24370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23880.log 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.24370.log.gz 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23106.log 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.23097.log.gz 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21532.log 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23880.log: /var/log/ceph/ceph-client.admin.23106.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.23880.log.gz 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22780.log 2024-09-21T23:19:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21532.log: 96.4% -- replaced with /var/log/ceph/ceph-client.admin.23106.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22111.log 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.21532.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22434.log 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22780.log: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.22780.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21980.log 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.22111.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.22434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23460.log 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.22434.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.21980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23950.log 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.21980.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23460.log: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.23460.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23950.log: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.23950.log.gz 2024-09-21T23:19:30.620 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T23:19:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m0.074s 2024-09-21T23:19:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m0.130s 2024-09-21T23:19:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m0.050s 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32135.log 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.32135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36931.log 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34034.log 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36931.log.gz 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.34034.log: -- /var/log/ceph/ceph-client.admin.38414.log 2024-09-21T23:19:30.643 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34034.log.gz 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38579.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42162.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38414.log.gz 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38579.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38579.log.gz -5 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33034.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40841.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42162.log.gz 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35876.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33034.log.gz 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38478.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40841.log.gz 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35876.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.35876.log.gz -- 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.28695.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33949.log 2024-09-21T23:19:30.644 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38478.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34526.log 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28695.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40253.log 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33949.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.32135.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36340.log 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34526.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34230.log 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40253.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36340.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33607.log 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25190.log 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34230.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33607.log.gz 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21428.log 2024-09-21T23:19:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.25190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41610.log 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25190.log.gz 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.tmp-client.admin.21428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40116.log 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21428.log.gz 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41610.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41610.log.gz 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33376.log 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40116.log.gz 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42706.log 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33376.log.gz 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39741.log 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42706.log.gz 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28860.log 0.0% 2024-09-21T23:19:30.646 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39741.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28014.log 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.28860.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42758.log 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28860.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28014.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40601.log 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36033.log 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42758.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34152.log 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40601.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33155.log 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36033.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33515.log 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34152.log.gz 2024-09-21T23:19:30.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33155.log.gz 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25073.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41656.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33515.log.gz 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.25073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37167.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr: 28.8% -- replaced with /var/log/ceph/ceph-client.admin.25073.log.gz 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37936.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41656.log.gz 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33263.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37167.log.gz 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37936.log.gz 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28676.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33263.log.gz 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36532.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40572.log 2024-09-21T23:19:30.648 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28676.log.gz 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36532.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.36532.log.gz 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42882.log 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39596.log 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40572.log.gz 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42882.log.gz 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43434.log 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39596.log: /var/log/ceph/ceph-client.admin.43616.log 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39596.log.gz 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43434.log.gz 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43194.log 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.43616.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39673.log 2024-09-21T23:19:30.649 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43194.log.gz 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29593.log 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39673.log.gz 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41652.log 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29593.log.gz 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33807.log 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41652.log.gz 2024-09-21T23:19:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41538.log 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42878.log 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33807.log: 0.0%/var/log/ceph/ceph-client.admin.41538.log: -- replaced with /var/log/ceph/ceph-client.admin.33807.log.gz 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.41538.log.gz 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.43189.log 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28521.log 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42878.log.gz 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43189.log.gz 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42942.log 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35562.log 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28521.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28521.log.gz 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29088.log 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42942.log.gz 2024-09-21T23:19:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42680.log 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35562.log.gz 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33999.log 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29088.log.gz 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40766.log 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42680.log.gz 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33999.log.gz 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41644.log 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33876.log 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40766.log.gz 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35647.log 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41644.log.gz 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33876.log.gz 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36280.log 2024-09-21T23:19:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28984.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35647.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28884.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36280.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28984.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28701.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29539.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28884.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36653.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28701.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29539.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36891.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37331.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36653.log: /var/log/ceph/ceph-client.admin.36891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36653.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41761.log 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36891.log.gz 2024-09-21T23:19:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37331.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.37331.log.gz -- 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.41094.log 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33217.log 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41761.log.gz 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36476.log 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41094.log.gz 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36152.log 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33217.log.gz 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36201.log 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36476.log.gz 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36152.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.36152.log.gz 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36854.log 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37503.log 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36201.log.gz 2024-09-21T23:19:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43330.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36854.log.gz 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36102.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37503.log.gz 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43330.log.gz 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43184.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36102.log.gz 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gz 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29468.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43184.log.gz 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29468.log.gz 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38355.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz -5 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36001.log 2024-09-21T23:19:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28828.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36001.log.gz 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38908.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42466.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28828.log.gz 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42466.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.30519.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42466.log.gz 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28832.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.30519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36314.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43642.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28832.log.gz 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38314.log 2024-09-21T23:19:30.656 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36314.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37910.log 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43642.log: /var/log/ceph/ceph-client.admin.38314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43642.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38314.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37744.log 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29211.log 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37910.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32981.log 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37744.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37785.log 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29211.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40311.log 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.32981.log: /var/log/ceph/ceph-client.admin.37785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41034.log 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32981.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37785.log.gz 2024-09-21T23:19:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40311.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28393.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40311.log.gz 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36534.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41034.log.gz 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43512.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28393.log.gz 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39427.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36534.log.gz 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43512.log.gz 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37430.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38706.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39427.log.gz 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37381.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37430.log.gz 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-09-21T23:19:30.658 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% 95.3% -- replaced with /var/log/ceph/objectstore_tool.30519.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38706.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37381.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31731.log 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39927.log 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.31731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42870.log 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31731.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39927.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34407.log 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42870.log.gz 2024-09-21T23:19:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43193.log 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35963.log 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34407.log.gz 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39199.log 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43193.log.gz 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34079.log 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35963.log.gz 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29373.log 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39199.log.gz 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29329.log 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34079.log.gz 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35467.log 2024-09-21T23:19:30.660 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29373.log.gz 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39980.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29329.log.gz 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35467.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35467.log.gz 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37535.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39980.log.gz 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42523.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30106.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37535.log.gz 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39107.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42523.log.gz 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.30106.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.30106.log.gz --verbose 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr: -- /var/log/ceph/ceph-client.admin.40164.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42549.log 2024-09-21T23:19:30.661 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39107.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38760.log 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40164.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41584.log 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42549.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33914.log 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38760.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39639.log 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41584.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37676.log 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33914.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33718.log 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39639.log.gz 2024-09-21T23:19:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33803.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37676.log.gz 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33112.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33718.log.gz 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43746.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33803.log.gz 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38123.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33112.log.gz 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37143.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43746.log.gz 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36734.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38123.log.gz 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36235.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37143.log.gz 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34145.log 2024-09-21T23:19:30.663 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36734.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43382.log 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36235.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43020.log 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34145.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29140.log 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43382.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37997.log 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43020.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29431.log 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29140.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29036.log 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37997.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29431.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29431.log.gz 2024-09-21T23:19:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29036.log.gz 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28836.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40079.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36409.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28836.log.gz 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31322.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40079.log: /var/log/ceph/ceph-client.admin.36409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40079.log.gz 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36409.log.gz 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39889.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.31322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43039.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39889.log.gz 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43039.log.gz 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29062.log 2024-09-21T23:19:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39360.log 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29062.log.gz 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28532.log 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43824.log 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39360.log.gz 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28532.log.gz 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41996.log 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33684.log 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43824.log.gz 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41996.log.gz 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40671.log 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36736.log 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33684.log.gz 2024-09-21T23:19:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40671.log.gz 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39798.log 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr: 95.2%/var/log/ceph/ceph-client.admin.36736.log: -- replaced with /var/log/ceph/objectstore_tool.31322.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33183.log 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36736.log.gz 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37531.log 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38227.log 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33183.log.gz 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34558.log 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37531.log.gz 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35750.log 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38227.log.gz 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41798.log 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34558.log.gz 2024-09-21T23:19:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28535.log 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35750.log: 0.0%/var/log/ceph/ceph-client.admin.41798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28664.log 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41798.log.gz 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35750.log.gz 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28535.log.gz 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39155.log 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28664.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28664.log.gz 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39155.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39155.log.gz 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43538.log 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35677.log 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.43538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42810.log 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43538.log.gz 2024-09-21T23:19:30.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35677.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35677.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40435.log 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37030.log 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42810.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40435.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43191.log 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37030.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43191.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43720.log 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43191.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28438.log 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43720.log.gz 2024-09-21T23:19:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gz 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39402.log 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28438.log.gz 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25274.log 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39003.log 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39402.log.gz 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.25274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41295.log 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25274.log.gz 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39003.log.gz 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42440.log 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41295.log.gz 2024-09-21T23:19:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40993.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42440.log.gz 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40932.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42881.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40993.log.gz 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43030.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40932.log.gz 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42881.log.gz 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42497.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39251.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43030.log.gz 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42383.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42497.log.gz 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33039.log 2024-09-21T23:19:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39251.log: /var/log/ceph/ceph-client.admin.42383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39251.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42383.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42602.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33039.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33486.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42602.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39532.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40473.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33486.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33568.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39532.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40473.log.gz 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41400.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28681.log 2024-09-21T23:19:30.672 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33568.log.gz 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37845.log 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41400.log.gz 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39703.log 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28681.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28681.log.gz/var/log/ceph/ceph-client.admin.37845.log: 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29114.log 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37845.log.gz 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39703.log.gz 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41030.log 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35390.log 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29114.log.gz 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41897.log 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41030.log.gz 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28490.log 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35390.log.gz/var/log/ceph/ceph-client.admin.41897.log: 2024-09-21T23:19:30.673 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42784.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41897.log.gz 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28490.log.gz 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29700.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40523.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42784.log.gz 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28536.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29700.log.gz 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40523.log.gz 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28464.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28536.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35823.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28536.log.gz 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39195.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28464.log.gz 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41938.log 2024-09-21T23:19:30.674 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35823.log.gz 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39195.log.gz 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34446.log 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36981.log 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41938.log.gz 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34446.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.34446.log.gz /var/log/ceph/ceph-client.admin.39070.log 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38469.log 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36981.log.gz 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33317.log 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39070.log.gz 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38469.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38469.log.gz 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40029.log 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39247.log 2024-09-21T23:19:30.675 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33317.log.gz 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5/var/log/ceph/ceph-client.admin.40029.log: --verbose -- /var/log/ceph/ceph-client.admin.34075.log 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40029.log.gz 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39247.log.gz 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43694.log 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37294.log 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34075.log.gz 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36830.log 0.0% 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43694.log.gz 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33211.log 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37294.log.gz 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42298.log 2024-09-21T23:19:30.676 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36830.log.gz 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33211.log.gz 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42157.log 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28418.log 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42298.log.gz 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43668.log 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42157.log.gz 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28418.log.gz 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40807.log 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41372.log 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43668.log.gz 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-09-21T23:19:30.677 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40807.log.gz 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42576.log 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41372.log.gz/var/log/ceph/ceph-client.admin.41860.log: 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43852.log 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42576.log.gz 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42020.log 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36530.log 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43852.log.gz 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41152.log 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28529.log 2024-09-21T23:19:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36530.log: /var/log/ceph/ceph-client.admin.41152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36530.log.gz 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41152.log.gz 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35751.log 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38912.log 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28529.log.gz 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40315.log 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35751.log.gz 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38912.log.gz 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43027.log 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39568.log 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40315.log.gz 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43027.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.42414.log 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43027.log.gz 2024-09-21T23:19:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34277.log 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39568.log: /var/log/ceph/ceph-client.admin.42414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36681.log 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42414.log.gz 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39568.log.gz 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.34277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34277.log.gz 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42095.log 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35554.log 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36681.log.gz 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35849.log 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42095.log.gz 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35554.log.gz 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37789.log 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39326.log 2024-09-21T23:19:30.680 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35849.log.gz 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43408.log 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37789.log.gz 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29210.log 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39326.log.gz 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38641.log 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43408.log.gz 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43036.log 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.38641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40906.log 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29210.log.gz 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38641.log.gz/var/log/ceph/ceph-client.admin.43036.log: 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43036.log.gz 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29214.log 2024-09-21T23:19:30.681 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41271.log 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40906.log.gz 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.29214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32761.log 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29214.log.gz 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41497.log 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41271.log: /var/log/ceph/ceph-client.admin.32761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41271.log.gz 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32761.log.gz 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43187.log 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35608.log 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41497.log.gz 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36804.log 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43187.log.gz 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42272.log 2024-09-21T23:19:30.682 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35608.log: /var/log/ceph/ceph-client.admin.36804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35608.log.gz 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36804.log.gz 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33743.log 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28530.log 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42272.log.gz 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.33743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41245.log 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33743.log.gz 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39044.log 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.28530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28530.log.gz 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38617.log 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41245.log.gz 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.39044.log: /var/log/ceph/ceph-client.admin.38617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39044.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38617.log.gz 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-21T23:19:30.683 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz 2024-09-21T23:19:30.697 INFO:teuthology.orchestra.run.smithi144.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T23:19:30.749 INFO:teuthology.orchestra.run.smithi053.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T23:19:30.799 INFO:teuthology.orchestra.run.smithi144.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T23:19:30.884 INFO:teuthology.orchestra.run.smithi053.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T23:19:30.940 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-21T23:19:34.798 INFO:teuthology.orchestra.run.smithi053.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T23:19:35.779 INFO:teuthology.orchestra.run.smithi053.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T23:19:36.515 INFO:teuthology.orchestra.run.smithi144.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T23:19:37.928 INFO:teuthology.orchestra.run.smithi144.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T23:19:37.930 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-21T23:19:37.930 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m7.460s 2024-09-21T23:19:37.930 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m13.535s 2024-09-21T23:19:37.930 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.572s 2024-09-21T23:19:38.363 INFO:teuthology.orchestra.run.smithi053.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T23:19:38.364 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-21T23:19:38.365 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m7.923s 2024-09-21T23:19:38.365 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m17.691s 2024-09-21T23:19:38.365 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.772s 2024-09-21T23:19:38.366 INFO:tasks.ceph:Archiving logs... 2024-09-21T23:19:38.366 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi053/log 2024-09-21T23:19:38.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:19:40.294 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi144/log 2024-09-21T23:19:40.295 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:19:41.691 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi170/log 2024-09-21T23:19:41.692 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T23:19:41.812 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T23:19:41.834 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T23:19:41.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:19:41.836 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:19:41.839 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T23:19:41.922 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T23:19:41.930 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T23:19:41.931 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:19:41.933 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:19:41.936 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T23:19:41.945 INFO:teuthology.orchestra.run.smithi053.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:19:41.945 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================== 2024-09-21T23:19:41.945 INFO:teuthology.orchestra.run.smithi053.stdout:*hv01.front.sepi 67.205.162.81 3 u 59 64 177 0.058 -3.203 1.483 2024-09-21T23:19:41.945 INFO:teuthology.orchestra.run.smithi053.stdout:+hv02.front.sepi 63.231.80.2 3 u 65 64 177 0.067 +0.448 1.070 2024-09-21T23:19:41.945 INFO:teuthology.orchestra.run.smithi053.stdout:+hv03.front.sepi 74.6.168.72 3 u 55 64 177 0.074 -5.445 2.237 2024-09-21T23:19:41.945 INFO:teuthology.orchestra.run.smithi053.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:19:41.948 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:19:41.948 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-09-21T23:19:41.948 INFO:teuthology.orchestra.run.smithi170.stdout:*hv01.front.sepi 67.205.162.81 3 u 58 64 177 0.078 -0.873 1.177 2024-09-21T23:19:41.948 INFO:teuthology.orchestra.run.smithi170.stdout:+hv02.front.sepi 63.231.80.2 3 u 58 64 177 0.239 +1.842 0.957 2024-09-21T23:19:41.948 INFO:teuthology.orchestra.run.smithi170.stdout:+hv03.front.sepi 74.6.168.72 3 u - 64 377 0.085 -2.467 1.215 2024-09-21T23:19:41.948 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:19:41.950 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T23:19:41.950 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-09-21T23:19:41.950 INFO:teuthology.orchestra.run.smithi144.stdout:*hv01.front.sepi 67.205.162.81 3 u - 64 377 0.051 +0.006 0.847 2024-09-21T23:19:41.950 INFO:teuthology.orchestra.run.smithi144.stdout:+hv02.front.sepi 63.231.80.2 3 u 63 64 177 0.087 +3.244 0.636 2024-09-21T23:19:41.950 INFO:teuthology.orchestra.run.smithi144.stdout:+hv03.front.sepi 74.6.168.72 3 u 61 64 177 0.066 -1.427 0.867 2024-09-21T23:19:41.950 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T23:19:41.950 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T23:19:41.959 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T23:19:41.959 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T23:19:41.967 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T23:19:41.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T23:19:41.986 INFO:teuthology.task.internal:Duration was 922.910526 seconds 2024-09-21T23:19:41.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T23:19:41.996 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T23:19:41.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:19:41.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:19:42.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T23:19:42.039 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T23:19:42.039 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T23:19:42.039 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:19:42.083 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-09-21T23:19:42.083 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:19:42.094 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T23:19:42.094 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T23:19:42.104 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T23:19:42.104 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:19:42.127 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:19:42.139 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T23:19:42.175 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T23:19:42.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:19:42.203 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:19:42.211 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T23:19:42.289 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T23:19:42.298 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T23:19:42.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:19:42.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:19:42.321 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T23:19:42.333 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T23:19:42.343 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:19:42.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:19:42.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:19:42.367 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-09-21T23:19:42.370 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-09-21T23:19:42.385 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-09-21T23:19:42.476 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:19:42.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:19:42.482 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:19:42.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:19:42.487 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T23:19:42.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T23:19:42.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T23:19:42.533 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T23:19:42.534 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi053 2024-09-21T23:19:42.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:19:42.556 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi144 2024-09-21T23:19:42.556 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:19:42.577 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914511/remote/smithi170 2024-09-21T23:19:42.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T23:19:42.602 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T23:19:42.603 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:19:42.605 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:19:42.619 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T23:19:42.651 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T23:19:42.662 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T23:19:42.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T23:19:42.672 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T23:19:42.672 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:19:42.674 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:19:42.676 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T23:19:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:19 /home/ubuntu/cephtest 2024-09-21T23:19:42.680 INFO:teuthology.orchestra.run.smithi144.stdout: 655474 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:19 /home/ubuntu/cephtest 2024-09-21T23:19:42.700 INFO:teuthology.orchestra.run.smithi170.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 23:19 /home/ubuntu/cephtest 2024-09-21T23:19:42.701 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T23:19:42.711 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T23:19:42.732 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 922.9105262756348 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T23:19:42.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T23:19:42.823 INFO:teuthology.run:pass